
THE 

10th CONFERENCE 
ON 

COMPUTER 
APPLICATIONS 
IN 

RADIOLOGY 

AND 

4th CONFERENCE 
ON 

COMPUTER 
ASSISTED 
RADIOLOGY 



COMPUTER 

APPLICATIONS 

TO 

ASSIST 
RADIOLOGY 



ANAHEIM 
CALIFORNIA 
June 13-16, 1990 



XjM 90 




Edited by 

Ronald L. Arenson, M.D. 



Professor of Radiology 

University of Pennsylvania School of Medicine 
Associate Chairman (Clinical Services) 
Department of Radiology 



Hospital of the University of Pennsylvania 
Philadelphia, PA 

Richard M. Friedenherg, M.D. 

Professor and Chairman 
Department of Radiological Sciences 
University of California, Irvine 
Orange, CA 



Symposia Foundation 



A Pubfic Nor}- Profit Organization Dedicated to the Health Cafe of the Public 



Published by 




PARTICIPATING ORGANIZATIONS 

American Association for Medical Systems and Informatics 

American Association of Physicists in Medicine 

Department of Radiology 
University of Pennsylvania Medical Center 

Department of Radiology and Continuing Medical Education, 
University of California, Irvine 

National Electrical Manufacturers Association 

Society for Photo/Instrumentation Engineers 



This book produced directly from camera ready 
mechanicals furnished by the authors. 



©1990 by Symposia Foundation 

All rights reserved. No part of this publication may be reproduced or trans- 
mitted in any form or by any means, electronic or mechanical, including 
photocopy, recording or any information storage and retrieval system, 
without permission in writing from the publisher. 

Symposia Foundation 
P.O. Box 2107 
Carlsbad, CA 92008 

Library of Congress Catalog Card Number 90-643663 
International Standard Book Number 0-88372-005-1 

Printed in the United States of America 



ORGANIZING COMMITTEE FOR 



COURSE DIRECTOR 

Ronald L. Arenson, M.D. 



Course Co-Director 

Richard M. Friedenberg, M.D. 



SCIENTIFIC SESSION CO-CHAIRMEN 



Clinical 

Ronald L. Arenson, M.D. 
Richard M. Friedenberg, M.D. 



Technical 

Roger A. Bauman, M.D. 
Jack Sklansky, Ph.D. 



Workshop Co-Chairmen 

Robert M. Allman, M.D. 
Henry A. Swett, M.D. 



Technical Exhibits Chairman 

Richard M. Friedenberg, M.D. 



International Coordinator 

Heinz U. Lemke, Ph.D. 



SPONSORING ORGANIZATIONS 

AT&T Medical Diagnostic Systems 
CEMAX, Inc. 
Columbia Scientific 
Computer Design & Applications, Inc. 
Dalcon Computer Systems 
Dictaphone Corporation 
Digital Equipment Corporation 

Dimensional Medicine, Inc. 
Dynamic Digital Displays, Inc. 
E.I. DuPont de Nemours & Company 
Fibronics International 
G.E. Medical Systems 
Health Information Systems 
Hewlett-Packard 
Kontron Electronics, Inc. 
Kurzweil Applied Intelligence, Inc. 
Lanier Corporation 
Medx International, Ltd. 
Megascan, Inc. 
Panasonic Industrial Company 
Philips Medical Systems, Inc. 
Reality Imaging 
W.B. Saunders Company 
SMS 

Siemens Medical Systems, Inc. 
Sunquest Information Systems 
Toshiba America Medical Systems, Inc. 
Vicom Systems, Inc. 
Winthrop Pharmaceuticals 



vii 



PROGRAM COMMITTEE 



Session Chairs 

Albert Bakker, Ph.D. 
Thomas Budinger, Ph.D. 
Antonio Chiesa, M.D. 
Samuel J. Dwyer, III, Ph.D. 
Roland Felix, M.D. 
Elliot K. Fishman, M.D. 
Richard M. Friedenberg, M.D. 
Joseph Gitlin, D.P.H. 
Melvyn Greberman, M.D. 
Robert A. Greenes, M.D., Ph.D. 
GaborT. Herman, Ph.D. 
Tim B. Hunter, M.D. 
Herwig Imhof, M.D. 
John W. Loop, M.D. 
Bruce Rosen, M.D. 
Alan H. Rowberg, M.D. 
Roger H. Shannon, M.D. 
Jack Sklansky, Ph.D. 
Edward V. Staab, M.D. 
Frans van der Voorde, M.D. 
Gabriele von Voigt, Ph.D. 



Tutorials 

Ronald L. Arenson, M.D. 
Robert A. Greenes, M.D., Ph.D. 
H.K. Huang, Ph.D. 
R. Gilbert Jost, M.D. 
Heinz V. Lemke, Ph.D. 
Michael L. Rhodes, Ph.D. 
George W. Seeley, Ph.D. 
Henry A. Swett, M.D. 

Workshops 

Paul Fisher 

Robert A. Greenes, M.D., Ph.D. 
Al Griebenow 
GaborT. Herman, Ph.D. 
Steven C. Horii, M.D. 
H.K. Huang, Ph.D. 
R. Gilbert Jost, M.D. 
Pradeep Mutalik 
Carl E. Ravin, M.D. 
Sridhar Seshadri, M.S.E.E. 
Henry A. Swett, M.D. 
Michael W. Vannier, M.D. 



INTRODUCTION 



Background 

The Society for Computer Applications in Radiology (SCAR) was formed 
several years ago to provide a forum for the exchange of scientific infor- 
mation concerning the use of computers in Radiology. SCAR was created 
under the auspices of the Radiology Information Systems Consortium 
(RISC), a non-profit organization dedicated to stimulating the development 
of quality computer systems in our field. 

The Society, SCAR, serves individuals interested in computer applica- 
tions and wishing involvement in an organization dedicated to these pur- 
poses. The Journal of Digital Imaging (JDI) is the official journal of SCAR. 
RISC has institutional members, not individual members, and typically 
arranges contracts with commercial vendors for joint development efforts 
with member institutions. 

The American College of Radiology (ACR) sponsored eight Confer- 
ences on Computer Applications in Radiology every other year since the 
early 70's. Because of financial concerns, the ACR turned responsibility 
for the conferences over to RISC for the 9th Conference, held in Hilton Head 
in June 1988. The ACR co-sponsored that meeting. These conferences 
have played an important role in the revolution in the use of computers in 
Radiology, with dramatic improvements in the technology and applications 
seen at each meeting. The earliest meetings were dominated by proto- 
types of Radiology Information Systems (RIS) as well as early attempts at 
computer-assisted diagnosis. At the latest meetings, Picture Archiving and 
Communications Systems (PACS) and image processing, such as 3D recon- 
structions, have dominated the presentations. 

Computer Assisted Radiology (CAR) is the organization which has spon- 
sored three very successful CAR symposiums in Berlin (held every other 
year). Heinz U. Lemke, Ph.D., has organized and masterly led those 
efforts from the very beginning. The CAR meetings have been oriented 
more toward the technical side, but offered two simultaneous tracks, one 
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technical and one more clinical. CAR has demonstrated the strength of both 
the European academic institutions and the commercial vendors in these 
fields. CAR has been a major player in organizing the European leader- 
ship and has also managed to attract many international colleagues as well. 

What does all of this have to do with S/CAR '90? This symposium 
represents the merger of these two previously separate series of confer- 
ences. S/CAR '90 is the 10th Conference on Computer Applications in Radi- 
ology and the 4th Conference on Computer Assisted Radiology. We believe 
that we have combined the best of both approaches in this conference, with 
dual tracks, tutorials, workshops, tours and this hardbound book. By bring- 
ing the technical and clinical strengths together and converting these meet- 
ings into truly international events, the quality, impact and appeal have been 
raised substantially. This symposium attracted 32 commercial exhibitors, 
more than any other meeting of its kind. 

S/CAR '90 Program 

The program for S/CAR '90 and the contents of this text are divided into 
the following major categories: Management Information Systems, Picture 
Archiving and Communication Systems, Image Processing, and Teaching 
and Research. Each of these areas will be described below. 

Management Information Systems 

During the earliest ACR Conferences as described above, the first pro- 
totype RIS's were under development at several leading academic institu- 
tions. No viable commercial products existed. Over the years, successful 
implementations have been presented and the commercial development 
of good management systems blossomed. 

The presentations concerning management systems in these proceed- 
ings deal mostly with reporting systems and the application of artificial 
Intelligence (Al) techniques. Speech recognition systems have improved 
dramatically in the recent past and several papers demonstrate the cur- 
rent applicability in our field. The use of rule-based programming and 
so-called expert systems are presented in a series of papers, including man- 
aging the display of digital images and assisting diagnostic decisions. 
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Teleradiology, the transmission of images between two distant locations, 
could be considered part of PACS. These systems offer the opportunity to 
provide superior service to our referring colleagues at a time when such 
service is competitively most important. Advances in automated schedul- 
ing and integration with other systems in the hospital are also important 
topics today. 

Picture Archiving and Communications Systems 

PACS are expected to replace film for the acquisition, transmission, stor- 
age and display of medical images in the future. Although there has been 
much enthusiasm on the part of both academic and commercial developers, 
few systems are clinically in operation at this time. RIS development took 
a decade to reach the level of hundreds of installations. PACS may follow 
a similar pattern, unless they can be found to be cost-effective soon. 

Several institutions have developed prototype systems and a few have 
clinical trials in place. These experiences are presented in several papers 
and show considerable progress, even since the November meeting of the 
Radiological Society of North America (RSNA). Exciting new developments 
in the United States, Europe and Japan are presented, taking advantage 
of the latest improvements in technology. 

The coexistence of film and PACS has become more of a reality as 
experience in PACS grows. These hybrid transition approaches are well 
documented in several papers. The importance of standards, including the 
ACR-NEMA standard, is also presented. The critical nature of the design 
of the digital workstation is brought into focus in papers dealing with the 
workstation as well as in papers on clinical design and evaluation. 

Image Processing 

The term image processing is quite general, incorporating a variety of 
different mathematical and computational tasks. Although image processing 
techniques have been used for years, only recently have some of these 
ideas appeared to have real clinical significance. Three-dimensional (3D) 
presentation of computerized tomography (CT) is still being refined, but 
much more attention is now being given to magnetic resonance imaging 
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(MRI). As demonstrated in several papers, cardiac MRI applications demand 
3D and time-motion presentations in order to be fully understood. 

Surgical planning continues to benefit from a variety of image process- 
ing techniques. In addition to facial reconstruction, the implantation of 
orthopedic prostheses is facilitated significantly. In addition, image proc- 
essing is beginning to show promise in other areas such as cholangiogra- 
phy, chest examinations and mammography. 

Teaching and Research 

Computer-assisted instruction (CAI) has been applied in Radiology for 
years, but has not yet realized widespread acceptance. The difficulties in 
course preparation, the cost of equipment and the limited experience on 
the part of radiologists have all contributed to the lack of success to date. 
As can be seen in the papers involving computers in teaching, the combi- 
nation of new software, cheaper and more powerful personal computers, 
and much more familiarity among radiologists is rapidly changing the way 
we teach our specialty. Although still important in education, the videodisc 
appears to be giving way to digital imaging techniques for medical images. 

Al and speech recognition also find their way into the teaching and 
research area, and the combination of PACS with personal workstations 
for teaching as well as clinical case presentation makes great sense. 

In the research section, MRI, ultrasound and mammography share the 
attention, with a potpourri of promising applications. Some of these develop- 
ments require supercomputers while others are managed on the desktop. 

Ronald L. Arenson, M.D. 

Professor of Radiology 

University of Pennsylvania School of Medicine 
Philadelphia, PA 



Future Symposia 

The following meetings are already well along in the planning stages. 
The CAR meetings will be held out of the U.S. every other year to alternate 
with the S/CAR meetings to be held in the U.S. Please mark your calen- 
dars for the following: 

Computer Assisted Radiology, CAR '91, July 3-6, 1991, 
Berlin, West Germany 

Symposium for Computer Assisted Radiology, S/CAR '92, 
June, 1992, U.S. site to be determined 

Computer Assisted Radiology, CAR '93, June 30-July 3, 1993, 
Berlin, West Germany 
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Film Labels: A New Look 

Tim B. Hunter and Robert Morrison 

Department of Radiology, University of Arizona Health Sciences Center, 
Tucson, AZ 



Every diagnostic image should be 
properly labelled with pertinent patient 
information, such as the patient f s name, 
hospital number, date of birth, and exam 
date. This is usually no problem with CT, 
ultrasound, Nuclear Medicine, and digital 
radiology studies because the imaging 
equipment is designed to automatically put 
the patient information in appropriate 
places on the films, typically across the 
top of each image or on a film header. 
However, most diagnostic radiology still 
involves the use of everyday x-ray films. 
These films usually are labelled by some 
type of film flashcard system. 
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No matter the method used, the majority 
of the labels always seem to be upside down 
or backwards when the film is put up for 
interpretation. A program, LABEL, was 
written to create a new font set containing 
upside down, tilted, and mirror image 
characters for a Digital Equipment 
Corporation (DEC) LA/75 dot matrix printer 
and an Okidata Microline 192 dot matrix 
printer. A film label was then designed to 
use this new font set to create film 
flashcard labels so they are immediately 
readable, no matter how the film is put up. 
The program was written in compiled BASIC 
for stand-alone systems and in MUMPS to be 
integrated into the IMAGES radiology image 
management system used at the University 
Medical Center in Tucson, Arizona. LABEL 
captures patient data from the IMAGES 
information system, and writes out an 
appropriate film flashcard using the DEC 



printer. 

LABEL also calculates the patient's age 
from his or her date or birth information* 
The age is then printed on the flashcard. 
it much easier to read the patient f s age off 
the film label than to calculate it mentally 
from the patient f s birth date. In the case 
of infants under a year of age the program 
calculates their age in months, weeks, or 
days. This has proven to be particularly 
helpful with neonates only a few days old. 

It is recommended that similar systems 
be instituted in other hospitals. This 
could take many forms. A special program 
could be written for one's own hospital 
information system so that it prints 
flashcard with special upside down and 
mirror image fonts. Or one could use the 
compiled BASIC form of the authors 1 program 
to control a dedicated microcomputer/printer 
setup to produce flashcards (write to the 
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authors for details concerning program 
availability) . Any program to print custom 
designed film flashcards should be easy to 
use and allow ready modification of the 
label through simple menu choices. Thus, 
the power of the computer not only allows 
you to have special fonts for high 
visibility of critical information , but it 
also allows you to change the label at will. 
You might want to move around information 
into different positions, delete certain 
information, bold information, italicize it, 
or underline it. All of this is possible 
with computer control of the film label. 

Of course, the important message here 
is not so much that of a computer program to 
design film labels but that film labels 
should be accurate, legible, and easy to 
read. The film label is a most important 
item and should not be neglected. The use 
of special fonts makes it less necessary to 



take a film down and turn it around to read 
the label. Good film labelling will not 
necessarily lead to good radiology, but bad 
film labelling with inattention to detail 
will inevitably lead to bad radiology. 
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In our large hospital-based radiology practice, difficulty in 
scheduling patient appointments is a highly visible problem. A 
written schedule for each subspeciality or examination suite may be 
accessed by only one appointment coordinator at a time* Our 
solution has been to provide a Radiology Information System (RIS)- 
based online scheduling capability, accessible from any location for 
any examination offered. Eliminating the need for a written ledger, 
it allows multiple users to simultaneously access centralized 
schedules. Retrieval of patient folders containing prior comparison 
examinations is automatic, and could be extended to a PACS to 
initiate pre-assembly of prior images in a workstation from an image 
archive. The schedule for any given day may be reviewed, allowing 
the radiologist to anticipate an examination that may require specific 
attention or imaging protocol 

Our scheduling module is integrated into the RIS developed 
at the University of Chicago, patterned after the Missouri Automated 
Radiology System (MARS). Given a statement of examinations 
requested, the module suggests the best available time and 
examination suite. Information is derived from logical templates 
defined for each examination and examination room. The user sees 
a graphic display of targeted rooms and times, and automatic 
scheduling is supplemented by a group of override capabilities. 
Templates accommodate differences between subspeciality practices, 
weekly schedules, weekends and holidays, and can be easily 
modified to adjust to changes in staffing or departmental policy. For 
example, a 1-hour 'slot* routinely reserved each morning for adult 
outpatient CT-guided biopsy procedures may automatically be 
released for general use if a biopsy has not been scheduled within 24 
hours of the time in question. The module avoids scheduling 
conflicting times for a given patient, and a further modification will 
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avoid conflicts from improper examination sequencing relative to 
contrast material retention or iodine usage. 

Patient throughput is enhanced because patients are 
preiegistered at the time of scheduling. If additional demographic or 
billing information is needed, it may be obtained prior to the 
patient's arrival in the Radiology Department and the patient may 
proceed more quickly to the examination room. Since the 
implementation of the computerized scheduling module in GI and 
CT, a greater efficiency of room utilization has increased the number 
of patients examined. The ease of scheduling an appointment has 
increased, probably because several appointment coordinators are 
able to access the centralized schedule simultaneously. Without 
increasing appointment coordinator staffing, the hours during which 
calls for scheduling appointments may be taken has been greatly 
extended. Any day's schedule may be viewed from any Radiology 
Information System terminal, including home PC's via a modem, to 
enable radiologists to more efficiently plan their day. Scheduled 
invasive procedures are continuously displayed at several locations 
throughout the department, updated every five minutes 
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INTRODUCTION 

Balancing the preferences and needs of the individual 
radiologists with the requirements of a group practice 
can be an unmanageable and potentially unsolvable 
problem. With the median size of private group practices 
in the United States at almost ten radiologists, the time 
required for performing this scheduling task manually 
appears to increase geometrically with the number of 
radiologists* It is hard to justify the costs associated 
with a radiologist diverted from other tasks or use of 
time to perform this one, with the availability of the 
program described in this paper. 

Over the past ten years, we have developed a rules-based 
expert system, CALENDOC , that can be customized to a 
particular practice. From the specific requests of the 
radiologists for time-off preferences, conflict-free 
multiple call rotation schedules and a prioritized daily 
work schedule are generated. 

Initializing the program for a specific practice can take 
a few hours. Entering the time-period specific informa- 
tion through the screen driven data editor is a much 
shorter process. Modifying the data is trivial. 
CALENDOC can then take that information and generate a 
complete schedule for up to twenty radiologists for an 
entire year in about one minute, depending upon the 
micro-processor type and speed in any IBM/MSDOS com- 
patible system. 
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BACKGROUND 

Radiologist scheduling is perfect for attack by a 
constraint or rules-based expert system. This scheduling 
task is usually performed manually, sometimes with some 
computer aided word-processing or spreadsheet format. 
But the rules, "the whys and the whens," are all done in 
someone's head, usually of one of the radiologists. For 
any given practice, the rules are usually very compli- 
cated. But in fact, there are more similarities than 
differences from practice to practice, and even from 
radiology to other specialties. 

We started in 1980 with a program built specifically for 
our practice. No commercially available programs existed 
at that time. In 1986 a program was reported from the 
Mallinckrodt Institute of Radiology. (1) Also, we have 
evaluated a product developed for anesthesia groups. (2) 
Two years ago, modifications were made to generalize the 
program to potentially any similar practice. 

One of the instigating factors in initiating this project 
was the popularization of the Pascal programming lan- 
guage. It had a feature that rendered it perfect for 
this task: the only widely used language that had 
provisions for scalar variables and set theory operations 
on those variables. Concepts like the whole set of 
radiologists, the set qualified to take first call or 
neuro call, the set of radiologists on vacation this week 
or next week, the set available to work this week, etc. 
form the core of the logic of our algorithm. 

PLEASE ENTER THE NUMBER OF DOCTORS (5-20): 18 
PLEASE ENTER THE NUMBER OF LOCATIONS (4-X5) : 13 
ENTER THE LINEUP OF DOCTORS USING THREE LETTER CODES: 
ANG AVR BRO FEL FER HLZ JMS LI MOO MUK MWD NYB PAR PET SWD WHI 2IB LOC 

Press <esc> to continue 



Figure 1. Query and input screen for basic group 
information. 
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ALGORITHM DESCRIPTION 



There are two parts of the scheduling problem. The first 
is the determination of major call - that is call taken a 
week or weekend at a time. The second is the determina- 
tion of who works where on any giveq day. These are very 
separate and almost unrelated problems. 

Major Call: 

Four pieces of input data determine the solution for 
major call: 



1. which radiologists are eligible to take a 
specific call rotation - Figure (2) 

2. the vacation/week off requests* - Figure (3) 

3. special requests for weekends off 

4. assigned weekend call for holidays or other 
purposes. 



(*These are determined by a manual process external to 
the program in keeping with the practice rules, not to 
exceed a maximum number that would leave jobslots 
uncovered . ) 

ENTER THE LINEUP OF THOSE AVAILABLE FOR FIRST CALL. THOSE DOCTORS 
PLACED AT THE BEGINNING OF THE LINEUP WILL RECEIVE MORE ASSIGNMENTS. 

2 DOCTORS WILL RECEIVE 3 CALL WEEKENDS AND 4 DOCTORS WILL RECEIVE 
4 CALL WEEKENDS. 

USE THE THREE LETTER CODES: 

PAR LI MUK MOO FER SWD 

Press <esc> to continue 

Figure 2. Weekend call roster information screen. 

ENTER THE VACATION SCHEDULE OF THE PRACTICE BY WEEKS. ALLOW 
NO MORE THAN FOUR DOCTORS OFF PER WEEK AND USE THE THREE 
LETTER CODES TO DENOTE THE DOCTORS: 

WEEK 1: FER ZIB 
WEEK 2: FEL 
WEEK 3: MOO AVR ANG 
WEEK 4: HLZ MWD SWD 
WEEK 5: ANG NYB 
WEEK 6: FER ZIB 
WEEK 7: HLZ MUK WHI 
WEEK 8: JMS NYB FEL 
WEEK 9: AVR JMS LI 
WEEK 10: MWD SWD PAR 
WEEK 11: 

WEEK 12: HLZ NYB WHI 

WEEK 13: BRO LI JMS Press <esc> to continue 

Figure 3. Vacation selection screen. 
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From these data, a major call schedule can be determined, 
for multiple call rotations. The algorithm that we have 
been using for a number of years works well. The goal of 
this algorithm is to maximize the spacing between on-call 
weeks or weekends for the greatest number of par- 
ticipants, and to absolutely obey certain prohibitions 
which include: 

1. on-call the weekend after a week of time off 

2. on-call the weekend beginning a week of time off 

3. on-call a requested weekend off 

4. on-call two weeks in a row 

5. on two different call rotations at once. 

The algorithm works as follows: The assigned calls are 
inserted into the schedule array, as are the requested 
weekends off. The vacation requests are inserted. The 
remaining call slots are then filled in rotation by those 
eligible without regard to the constraints (2) and (3) 
above. 

Then a group of procedures sweeps through each call 
schedule radiologist by radiologist, week by week, and 
checks for "eligibility" for that particular weekend, and 
if a violation is found, starts searching the nearest 
weeks, and if necessary the entire schedule, forward and 
backward, for a permissible "swap". If such a swap to 
another week with another radiologist is found, then it 
is made. Assigned weekends are not swappable. Each 
entire specific call schedule is massaged this way, first 
with at least four week spacing between call intervals, 
then three week if no solution can be found, and then two 
week, i.e. every other week intervals. An error is 
generated if no "clean" (no violations of constraint) 
solution can be found. 

Weeknight Call Rotations (Minor Call) : 

One night at a time weeknight call rotations are imple- 
mented, and are dependent primarily on not being on a 
week off, and not on day off either the day of call or 
the day after. 
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DAILY WORK ASSIGNMENTS 

Daily work assignments are made with a scheme best 
described as two-dimensional prioritization. The two 
basic concepts are that certain daily assignments are 
more important or sensitive to the practice in their 
staffing and can be ranked; and that for each daily work 
slot there is a prioritization of who works there. In 
fact, in this era of subspecialization even in private 
practices, certain radiologists may not be eligible for 
particular slots. See Figure (4). 

Note that the ranking is from the perspective of the 
needs of the practice, not the preferences of the 
radiologist. Individual preferences are accomodated 
indirectly by not including radiologists in rosters for 
locations that they do not want to work, and by a higher 
priority ranking in those locations that they desire, if 
congruent with the desires of the group. 

ENTER THE PRIORITY ROSTERS FOR THE WORK SITES 

AS WELL AS ANY ROTATION VALUE FOR THE ROSTERS IN THE 

FORM Qn WHERE n VARIES FROM 0-13 AND QO IMPLIES NO 

ROTATION. UTILIZE THE THREE LETTER CODES FOR 

THE LOCATIONS AS WELL AS THE DOCTORS. 

WEEK 01: (FOR WEEK 1 PLEASE ESTABLISH ALL WORK LOCATIONS) 

LOCATION: SJP ROTATION: Q 2 
HLZ SWD FER LI 

LOCATION: IC1 ROTATION: Q 0 
WHI HLZ MUK FER ZIB JMS 
LOCATION: IC2 ROTATION: Q 0 
HLZ MUK FER JMS ZIB ANG 
LOCATION: MOl ROTATION: Q 0 
MWD FER MUK JMS ZIB 
LOCATION: M02 ROTATION: Q 0 
FER AVR JMS MUK LI ANG ZIB 

Press <esc> to continue 

Figure 4. Daily work slot priority roster first screen. 

One other task that is performed before actually 
assigning the daily work slots is to distribute 
additional days off in some equitable or proportional 
manner. These additional free days arise from less than 
a maximum permissible number of radiologists being on 
vacation or meeting weeks. Our practice allows up to 
three radiologists off each week, but some weeks are 
chosen by only one or two radiologists, or not chosen by 
anyone. CALENDOC allows and encourages that kind of 
flexibility. 
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One minor point is that Mondays and Fridays off, which 
generate three day weekends (a valuable commodity) are 
kept track of separately from midweek days. Also, any 
specially requested days off are charged to the ap- 
propriate radiologist against his/her allocated number. 
CALENDOC also has provisions for occasional administra- 
tive days off, which are not charged to an individual's 
allocation of days off, but which do remove a radiologist 
from the set available for the daily work slots. 

Finally, two additional flexibilities are provided. A 
radiologist can be allocated extra individual days off by 
trading for a full week off. A half-day off rotation 
also can be enabled and equalized. 

At this point we should be left with just enough radiolo- 
gists to do the day ! s work. The jobs are filled in the 
order of priority described above, as defined in the 
datafile. Each job is filled from the job slot roster, 
defined by query and input in the datafile, by the first 
available radiologist in that roster (not on vacation or 
day off, and not yet pulled for a higher priority slot). 
The rosters must have deep enough "bench strength", 
particularly for the lower priority positions, to make 
sure that all slots get filled. Too many dayoff re- 
quests, or inablity to fill a job from the rosters 
generate error messages. 

Two last features are that a roster (for example for MR) 
can be made to rotate on a daily, weekly or any multiple 
of weeks basis so that the number one radiologist goes to 
the bottom, and all others move up one position; that any 
or all rosters can be modified beginning any week, and 
repetitively throughout the schedule. 

SCHEDULE OUTPUT 

The actual schedule is stored in an array defined by 
date, call and location. However the work schedule 
printout can be by location, or turned around by radiolo- 
gist, as shown in Figure 5. The radiologists usually 
prefer the second mode, while the technical and manage- 
ment staff prefer the first. 

Output includes a tally screen that summarizes type and 
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number of call assignnments, as well as days off, for 
each radiologist. 

SUMMARY 

We have describe a rules or constraint based expert 
system that is generic to the radiologist scheduling 
problem, yet can be customized for a particular practice. 
CALENDOC contains user-friendly query screens, and 
implements the set theory and other logic described 
above. It checks constraints without error that no human 
could reasonably be expected to perform. 

Future developments include improved tree-structured 
query screens, and an "editing" version to allow for 
expected or unexpected changes. 

CALENDOC allows practices and radiologists to schedule 
flexibly, reconciling individual needs with the require- 
ments of the practice, while guaranteeing full coverage. 
It is a powerful time-saver for medium to large groups. 
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Figure 5. Printed output by radiologist and location 
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Summary 

Radiology Information Systems (RIS) are usually developed 
as stand-alone departmental systems to serve the basic 
information needs of the radiology department. Coupling 
with the Hospital Information System (HIS) operational in 
the surrounding hospital is usually restricted to patient 
identification from the hospital's patient database and 
recording of examinations performed for billing purposes. 
However further integration of these systems is needed to 
improve the communication between the radiology department 
and the in- and out-patient departments in the hospital. 

1. Introduction 

Radiology departments are part of the surrounding hospital 
and have many lines of communication to and from the 
various in- and out-patient departments. In spite of this 
Radiology Information Systems (RIS) usually have only a 
meagre coupling with the Hospital Information System (HIS) 
operational in the hospital. The import of patient 
identification data from the HIS to the RIS and the export 
of data on procedures performed for billing purposes are 
quite common. In few cases order entry from the in- and 
out-patient departments is possible[l]. There are also 
some examples of report-printing in the in- and 
out-patient departments. 

In which way can a further HIS/RIS integration improve the 
communication between the radiology department and other 
hospital departments and even to requesting physicians 
outside the hospital? In which way can this improved 
communication contribute to a better performance of the 
hospital as a whole? We will try to answer these questions 
by sharing our experiences in this field. We base our 
experiences on the BAZIS HIS. This HIS is built from about 
70 subsystems around a central hospital database[2]. The 
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BAZIS RIS, called RADI, is one of these subsystems, with 
functions for patient registration, examination 
scheduling, archive management, film-folder tracking, 
report generation and patient tracking. The BAZIS HIS is 
running in 40 Dutch hospitals, including 7 University 
hospitals, which have together about 20,000 acute beds. 
The BAZIS RIS is running in twenty hospitals. 
In the next sections we will give our experiences with 
HIS/RIS integration in the Leiden University Hospi tal(LUH) 
in relation to the improvement of the communication, 
planning and management of the flow of patients, reports 
and X-ray f ilm-folders[3,4] . We also try to indicate the 
direction of future developments in this field. 

2. HIS/RIS integration in appointment scheduling 
Communications with other departments and the planning of 
examinations can be improved in two ways: 

- to give the radiology department and in- and out-patient 
departments insight into each others appointments while 
making a new appointment for the same patient 

- to decentralize the appointment scheduling to the 
requesting physician or department. 

2.1 Scheduling supported by other scheduling data 

In the LUH both the radiology and out-patient departments 
make use of appointment scheduling functions and take 
information about other scheduled visits of the patient 
into account. Insight into each others appointments is 
found to be an advantage because it gives the user and the 
patient the opportunity: 

- to prevent the exposure of the patient to more than one 
heavy examination in one day 

- to combine the various appointments of a patient to 
avoid waste of time 

- to schedule the X-ray appointment a predefined period 
(e.g. 24 hours) in advance of an out-patient visit in 
order to guarantee the availability of the report before 
patient's visit to the out-patient department. 

2.2 Decentralized order entry 

As an experiment we authorized a small group of users of 
one of the internal medicine wards to make use of the 
radiology appointment scheduling functions for a limited 
set of examinations. This experiment was done to find out 
the user requirements for this decentralized function 
which are expected to differ from those of the radiology 
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personnel. Some of our experiences were[5]: 

- The possibility to choose an appointment date and time 
that best suits the patient is found to be an advantage. 

- Date and time of the appointment are registered 
immediately. Therefore coinciding appointments occur 
less frequently. 

- For some examinations the waiting time for an 
appointment is too long. In this case the radiology 
department is called by phone to schedule the patient in 
the overflow area of the agenda. 

- 45% of the requests for an examination is scheduled by 
the users in the ward. The remaining examinations 
require consultation with the radiologist or chief 
technician. 

- The radiology department experiences a decrease in 
contacts by telephone. 

- For the radiology department it sometimes is more 
difficult to realize a balanced workload. 

Future developments will be stimulated by the realisation 
of the BAZIS nursing information system. In this system a 
request for an appointment for e.g. a colon X-ray 
'explodes' into a series of actions and registrations[5] : 

- check on possible contra-indications and conflicts with 
other appointments for this patient 

- check on desired date and time in the agendas of the 
radiology department and registration of the appointment 

- identification of all nursing activities related to a 
colon X-ray and registration of these activities in the 
nursing workplan 

- registration of medication orders and initiation of 
transport of the patient to the radiology department. 

3. HIS/RIS integration in reporting 

As soon as the report is authorized it should be available 
for and sent to the requesting physician. There are four 
ways in which this could be supported by the integration 
of RIS and HIS: 

- The authorized report is sent immediately via the HIS 
terminal network to a printing device in the ward or 
out-patient department. 

- The authorized report is sent immediately to the 
information system of the requesting physician in- or 
out-side the hospital. 

- The requesting physician can ask the system for all the 
available reports with a selection on a certain period 
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or on certain examinations. 

- The existence of an authorized X-ray report can be 
reported to the in- and out-patient information system 
that keeps track of the progress of all patient's 
examinations, lab tests, etc* The requesting physician 
can, at the moment of his own choice, ask the system for 
all the available results. 

In the LUH reports are printed in the in- and out-patient 
departments immediately after authorization of the report. 
The printing of authorized reports in the typing area of 
the radiology department is almost redundant: 

- All the reports are on-line available for a period of 
more than 5 years. 

- Users throughout the hospital have access to the on-line 
medical record system, called PATIENT, to retrieve 
information about lab results, including pathology, 
radiology examinations and reports, admission and 
discharge diagnosis, surgery information, radiotherapy 
treatments, blood transfusions and medication 
prescriptions. PATIENT is used by users throughout the 
hospital, including radiologists to support their 
diagnostic process. 

- The development of a system called MIRAGE[6] that 
supports the transport of ADT data, radiology reports 
and lab results via a public data network to the 
information system of the patient's general 

practi tioner(GP) . This system is in use since April 1989 
by 5 hospitals to support the communications with 18 
GP's. 

We make use of a so-called EDI service (Electronic Data 
Interchange) to exchange data between the HIS and the GP 
information systems by means of agreed standard 
messages. These standards are based on the EDIFACT[7] 
standards used in the logistics environment for trade 
data exchange. We defined messages for ADT data, 
radiology reports and lab results. New messages will be 
defined for specialist letters, results of 
microbiology-tests and pathology. In 1990 we plan to 
extend the implementation of this system to 13 hospitals 
and more than 70 GP's. 

The total number of exchanged messages in 1989 was 
12,500. This number will increase in 1990 to more than 
50,000 messages. At this stage of the MIRAGE project the 
following conclusions can be made: 
. The network is reliable and the use of the EDIFACT 
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standard offers no technical problems 
• The gain of time of about 30 hours is not yet 

considered to be relevant by the participating GP's 
. GP's consider the integration of the communication 

into their information system to be the most important 

benefit . 

In the long run the reporting of X-ray examinations will 
become part of an out-patient information system with 
which the requesting user can keep control of the progress 
of lab tests and exams, and has access to the available 
results and reports. 

4. HIS/RIS integration and archive management 

The availability of old and new X-ray film folders is 

essential for: 

- the radiologist while reporting new images 

- the specialist preparing patient's out-patient visit 

- the specialist preceding and during the admission of a 
patient. 

In order to realize both a better planning of archive 
personnel workload and high service levels to the 
requesting specialists, requests for film folders have to 
be registered at the moment an appointment for X-ray 
examinations or an out-patient visit or the admission is 
scheduled. With this information the archive management 
system can generate information about which folders have 
to be transferred when to which departments. 
Before the introduction of the system departments used to 
send requests to the different file rooms in the hospital. 
In daily practice this is found to be an unstructured and 
unreliable way of communication: 

- requests arrive written on a variety of notes and forms, 
or orally either by telephone or in person 

- requests are sent to the wrong file room 

- requests are sent for patients that never visited one of 
the X-ray departments before 

- requests arrive too late. 

Integration of the request film folder function of the RIS 
in the HIS can yield a major improvement of the 
communication between the requesting departments and the 
file rooms, and an improved operation of the file rooms. 

In the LUH most of the in- and out-patient departments 
request film-folders via their HIS display terminals. For 
each archive location the archive management system 
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collects all the requests that have to be dealt with 
within a certain period. Only requests for film folders 
that are really available in the archive location are 
presented on a pull list in archive order. Requests for 
lended folders can be printed on a list at the borrowers 
location. The number of requests increased with the 
implementation of the decentralized request function. The 
users were interviewed [4] and were found to have positive 
experiences with the system: 

- The overview of all the available film folders with the 
possibility to point out the wanted folders encouraged 
the user to issue a request. 

- The required time for requesting has been reduced 
considerably, 

- On the other hand users became aware that they sometimes 
submitted a request for folders that should be already 
in their possession! 

- The users experienced the archive to be a reliable and 
fast supplier of the requested folders: the lead time 
was reduced considerably. This improved the relations 
between borrowers and archive staff. 

- The ratio of honoured/requested folders increased from 
80 to 98% due to the more accurate registration of the 
actual location of the folders. 

The archive personnel experienced: 

- a fast decrease in request forms, telephone calls and 
urgent requests 

- a more structured method of working and a more balanced 
workload 

- no request for folders not present in the file room 
occurred 

- less time-consumption in searching for lost folders 

- a considerable reduction of the number of lost folders 
Although the number of requests increased with about 10% 
to 1000 requests per day and the number of examinations 
increased with about 18% from 1982 until 1988, the 
permanent archive staff in the LUH remained constant at 
7.5 fte and the number of temporary staff was reduced from 
0.2 to zero. 

5. Conclusion 

Integration of a RIS in a HIS environment provides a 
splendid opportunity to contribute to the optimizing of 
the patient care process in the hospital as a whole in 
stead of the radiology department only. 
Decentralized functions for scheduling of X-ray 
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examinations, requests for film folders and insight into 
authorized reports: 

- improve the communication with the radiology department 

- improve the planning of the workload in the radiology 
department 

- realize higher service levels towards the in- and 
out-patient departments and to GP's 

- realize a better service to the patient. 

The direct availability of other patient related HIS data 
is a help in the diagnostic process of the radiologist. 
A major part of the workload( 10-40%) in the radiology 
departments of Dutch hospitals is triggered by general 
practitioners. This emphasizes the need for further 
integration of the HIS with the outside world, especially 
with GP' s-information systems. 
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Introduction 

A Computerized Radiology Management database 
System (CRMS) for Radiology has been in 
operation at the State University Hospital, 
Health Science Center in Brooklyn (HSCB) since 
November of 1987. The initial database 

system was a single user system* It is 
currently running under a local area network 
with nineteen workstations and will continue 
to grow with additional functions. 

Our Radiology department performs 52,000 
examinations annually including conventional 
X-Rays, nuclear medicine procedures, 
sonographies, CT, MRI and angiographic 
studies. The purpose of the CRMS was to 
enhance patient registration, to reduce 
patient registration errors, to automate 
patient ID flash cards, to facilitate archival 
operations, track film usage, generate 
statistic reports on a daily basis while 
compiling monthly and annual reports, to 
reduce lost films and to become a less labor 
intensive environment. 

Hardware Architecture 

The hardware of CRMS includes a file server, 
eleven Santa Clara diskless workstations, 
eight XT/AT workstations, a 1200 watts 
Uninterrupted Power Supply system (UPS) by 
American Power Conversion inc., a 60 megabyte 
cassette tape backup system from Emerald Inc., 
two network printers and local printers for 
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each workstation. The file server is an AT- 
compatible computer with an Intel 80286 
processor of 8 MHz clock speed and zero wait 
state. This file server consists of 2 megabyte 
memory, one 360k byte floppy disk drive and 
two 150 megabyte hard disks mirrored. The 
file server also includes an 8-bit ARCNET 
network interface card (NIC) from Standard 
Microsystem Inc and two 16 bit Ethernet-type 
NIC, the Ether link from 3Com Inc. The 
Etherlink uses the Carrier Sense Multiple 
Access with Collision Detection (CSMA/CD) 
access scheme with a 10 MHz baseband 
transmission. The ARCNET uses token-passing 
access scheme with a 2.5 MHz baseband 
transmission. The Santa Clara workstations 
are diskless XT computer with a 640k byte RAM 
memory, a built-in Ethernet NIC with a boot 
ROM. Each XT/ AT computers are IBM compatible 
with 640k byte RAM, a 360k byte floppy drive, 
a 20 megabyte hard disk and an 8 -bit ARCNET 
NIC manufactured by Standard Microsystem Inc 
also. All the diskless workstations are 
connected into two different linear bus 
systems to the file server with its two 
Etherlink NIC via RG-58 coaxial cable. Using 
the two separate linear bus topologies ensures 
continuous patient registrations even if one 
of the Ethernet NICs in the file server 
malfunctions. All eight XT/ATs are connected 
to an active hub using star topology as shown 
in Figure 1. The ARCNET NIC of file server is 
connected to the active hub. We decided to 
use the ARCNET for the XT/ATs is to allow 
comparison of speed, reliability, 
accessibility, ease of workstations addition 
between two networks. The UPS ensures the 
consistency of the commercial power and allows 
enough battery power to safely close all the 
files during a power outage. This UPS has the 
ability to monitor the commercial power line 
and shut down the file server safely when the 
commercial power sags below certain voltage or 
when there is a complete loss of the power. 
The installation of the UPS seems to have 
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alleviated previously experienced file 
corruptions among index files and data files. 
The first network printer is used to list each 
completed patient registration* The second 
network printer produces the patient 1 s ID 
flash cards and the corresponding patient f s 
index file card (this has been eliminated 
since the system has a proven reliability) . 
The Emerald cassette tape system is used to 
backup the patient records daily. With its 
software package, system backup can be done 
automatically at midnight when the file server 
is most likely idle. Local printers are used 
to print patient registration forms and 
statistical reports at each workstation. 

Software Architecture 

The network software, (the Novell Advanced 
Netware 286 SFT version 2.1) was chosen after 
a careful investigation among other networking 
softwares (e.g. 3Com, IBM PC network software 
etc.). We feel Netware provides better memory 
cache, disk cache and a sophisticated system 
fault tolerance (SFT) with time transaction 
system (TTS) . The memory caching and disk 
caching increase the speed of input/ output of 
files from the file server. The SFT supports 
disk mirroring and disk duplexing. Utilizing 
these two techniques, data are written onto 
two disks simultaneously so that one disk 
provides a mirror image of the other. This 
function also provides an on-line disk backup 
system; therefore, if one disk fails, the 
backup disk will continue to operate without 
downtime or the loss of data. TTS employs a 
scheme called read-after write verification to 
guarantee data integrity. Once the data is 
written, the system reads the data to make 
sure that it is properly saved. The SFT and 
TTS provide the added on-line security of our 
CRMS. 

The CRMS was written in Foxbase Multiuser 
Version 2.1. This database software package 
was chosen after benchmark studies on various 
database software including DBASE III Plus 
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with LanPack version 1.1, Paradox Version 2.0, 
and Clipper Version 2.1. The indexing speed 
and ease of programming in Foxbase Multiuser 
Version 2.1 appears to be more suitable for 
our database structure. 

Database File structure 

The CRMS consists of four major files: 
patient demographic, patient examinations, 
monthly patient registration and statistics. 
The patient demographic file contains 
patient's full name, sex, social security 
number (SSN) , date of birth (DOB) , address, 
hospital identification number and radiology 
medical record number (MRN) . In our 

institution, if the patient has a SSN, the 
last four digits of the SSN will be used as 
the MRN. However, if the patient is a 
foreigner or newborn , a MRN will be 
sequentially assigned to them by the 
registration clerk. This file is indexed in 
ascending order with the patient f s last name, 
MRN and his/her DOB. The patient examination 
file contains the patient's last name, MRN, 
his/her DOB, examination date, and the related 
examination code. Each examination date will 
represent one record of data entry. The 
monthly patient files contain the patient's 
last name, MRN, his/her DOB, examination type, 
site location where the examination is 
initiated, examination date, shift (day or 
night) , billing code, patient type, 
technologist who did the study and number of 
films used. In this file, each examination 
performed is regarded as a record. The 
patient type relates to the category of the 
patient that he/she belongs to; in-patient, 
employee, private, hospital clinic patient, 
and student. Various statistical files are 
generated from the monthly patient file based 
on examination procedure, diagnostic 
procedures, patient types, film usage and etc. 



Program Layout 
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The CRMS is a menu driven system divided 
into different functions: patient 
registration, patient search, film input, to 
modify daily input, patient demographic 
lookup, report generation, and system utility. 
Each function is written as a program module 
so that additional functions can be easily 
added . 

A. Patient registration : 

After choosing patient registration from the 
main menu, the registration menu is displayed. 
The registration clerk must type the patient's 
full name and last four digits of the social 
security number as MRN. The system will 
determine if this patient had previous 
examinations in our institution by searching 
and displaying all the possible patients that 
will match this patient's last name and MRN. 
The registration clerk decides if there is an 
accurate match and selects the match. If a 
match is chosen, the computer enters the 
patient's demographic information. If no 
match is found, a complete demographic form 
will appear on the screen. The registration 
clerk is required to fill in all the 
demographic information about the patient. 
Next the clerk enters the examination type and 
the corresponding examination code. Here, 
multiple examination types and examination 
codes can be entered. These examination types 
and the examination codes are cross checked. 
Once the correct examination types and codes 
are entered, the registration is complete and 
a record of data entry, a registration form, 
index card and a flash card are printed by the 
respective printers. 

Pre-registration can be done by simply 
changing the examination date. This module 
also allows any user at a workstation to check 
the list of patient registration according to 
examination type or patient name, or the 
entire list. 

B. Film entry 

Before the procedure is completed, each 
film is checked by a quality control 
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technologist. He/she enters the film usage 
for the study and the technologist initials 
performing the examination. The quality 
control technologist can list patients by 
name, by order of registration, or by 
examination type. Checking for the 

technologist initials or film usage on entry, 
the record is used to indicate the completion 
of examination. 

C. Search 

This program module allows a fileroom clerk 
to search for a patient and list out the 
examination history including the examination 
date and the types of examinations. The 
system can search the patient record based on 
name, MRN, SSN, and DOB or any combination of 
the listed information. Once a match is 
selected, the program lists the examination 
history of the match via a printer or on the 
monitor . 

D. Report generation 

This function can generate daily, monthly 
and annual reports. Reports are generated for 
each division (CT, MRI, US, Diagnostic, 
Special procedure, and Nuclear Medicine) . 
Each report totals the number of examinations 
performed by the examination code based on the 
type of patient, it also totals each 
subcategory as shown in Figure 2. All of the 
statistical reports related to procedure, type 
of examination and patient type are further 
divided into regular and pediatric patients 
(patient under 18 year old) . 

A separate film usage report is generated 
based on film size and type. 

E. Utilities 

This function allows the database manager to 
backup monthly patient files to floppy 
diskettes, restore monthly patient files from 
floppy diskettes, update examination codes 
including examination fees, modify monthly 
files, modify patient demographic data and 
examination data files, and back-loading of 
all the previous examinations into examination 
file from the manual card file system. 
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Cost Analysis 

The total costs of this system was less than 
$60,000. The detail costs are shown below. 



File Server includes 2 150 Megabyte 



hard disk & 2 Megabyte RAM 


$ 


4, 


,500 


11 Diskless Workstation 


$ 


9, 


,900 


8 AT Computer 


$ 


16, 


,000 


Cassette Tape backup System 


$ 


1, 


,200 


2 Ether link NIC 


$ 


1< 


, 000 


9 ARCNET NIC 


$ 


2, 


,250 


Active Hub 


$ 




800 


UPS 


$ 


1. 


,200 


20 printers 


$ 


10 , 


,000 


Novell Netware SFT V2.1 


$ 


3, 


,000 


Multiuser Foxbase V2.1 


$ 




500 


Cable and installation 


$ 


3, 


,000 


Total 


$ 


54, 


,300 



Conclusion 

The previous registration system required 
the registration clerk to rummage through the 
patient card files to check if the patient had 
previous examination performed in our 
institution. If the patient card was found , 
the requested studies are typed on the card. 
A patient may have many cards under his name 
because of misfiling, misspelling and change 
of name (married) . If the patient card was 
not found , a new card must be typed and filed. 
The registration form was typed next. When 
the technologist received the registration 
form, a flash card had to be handwritten 
before the examination could be done. Upon 
completion of the examination, the quality 
control technologist entered the completed 
study and film usage in a log book manually. 
The entire procedure took 7-9 minutes. The 
CRMS reduces this time down to 1 minute for 
the returning patient and 2 minutes for the 
new patient. The CRMS checks the validity of 
all the data entry, hence mistakes are 
minimized. Also, erroneous data are checked 
by the database manager and corrected. The 
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card files were centralized at the front 
desk, hence all patients were required to go 
to the front desk to register first before 
going to the respective division for the 
examination. With the CRMS, registration is 
done more efficiently by each division and the 
patient is not inconvenienced. Statistics on 
examination were done manually by counting 
from the log table in each division. This 
tedious report generation usually required the 
chief technologist in each division to spend 
days to get a monthly and annual reports and 
often the reports were not accurate. CRMS 
allows the database manager to generate each 
monthly report in less than 3 minutes and 
annual report in less than 10 minutes for the 
whole department. 

Although Ether link NIC uses a larger packet 
size (1024 byte) than ARCNET NIC (512 byte), 
this smaller packet size in ARCNET only 
affected the rate of transfer if data 
required to be sent is bigger than 512 byte. 
Also, the ARCNET transmits at a rate of 2.5 
MHz as compared to Etherlink f s transmit rate 
of 10 MHz. The ARCNET NIC is a much slower 
system, however, our system is not very busy 
and no appreciable difference between the two 
networks has been noticed. The I/O is not so 
busy that the transmission rate does not play 
an important factor in our configuration. One 
patient record is 490 bytes so the smaller 
packet size used in ARCNET NIC has no effect 
in our data transfer rate and ARCNET NIC is 
adequate for our CRMS. The major advantage of 
ARCNET NIC is the ease of adding more 
workstations and price. In order to add a new 
workstation for bus topology (Etherlink) , one 
has to locate the end of the bus run and add 
the new workstation to the end. However, 
using star topology (ARCNET NIC) , we can 
install an additional active hub to one of the 
ports of the previous active hub and eight 
more workstations can be added. This is major 
advantage for any system because the 
possibility to grow on any system is very 
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likely and is impossible to design a system to 
anticipate such growth before hand. 

The system has been very well received by 
everyone in the department. Training is 
accomplished in a few hours. An increase in 
efficiency and accuracy is evident. Barcoding 
is being added to help in film tracking and 
special nuclear medicine features are being 
added presently. A billing module is 
available, however our institution utilizes a 
separate billing system so this module is not 
activated. 
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2004 CT Scan - Head 34 


8 


11 


0 


53 


2204 CT Scan - Chest 9 


4 


3 


0 


16 


2261 CT Scan - Body 31 


27 


7 


0 


65 


2386 CT Scan - Neck 0 


0 


1 


0 


1 


2387 CT Scan - Upper Ext. 1 


0 


0 


0 


1 


2388 CT Scan - Lower Ext. 0 


1 


0 


0 


1 


2389 CT Scan Abd-Stereo 3 


8 


0 


0 


11 


2390 CT Needle Biopsy 1 


1 


0 


0 


2 


Total 
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Figure 2: Sample Report 
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Summary 

The Computerized Model developed jointly by TMG and IBM for 
cost analysis of planned installations of IBM Radiology 
Image/Information Management (RIM) systems was applied prior 
to the decision for the installation of RIM in three hospitals- The 
computed results for these hospitals indicate that the Model is a 
useful tool for the cost impact analysis necessary for justification of 
acquisition of a Spectrum* RIM- The projected cost savings 
resulting from RIM are such that the investment may pay back for 
itself in 2 to 4 years. It was also noted that the RIM cost savings 
constitute a substantial fraction (approximately 32%) of the total 
projected cost/benefit impact of a hypothetical installation of a 
complete PACS in these hospitals. 

"Spectrum (partnership of IBM Corporation and Baxter Healthcare 
Corporation) RIM, formerly called IBM RIM 

Introduction 

The cost justification for acquisition of new technologies in health 
care is an important and often deciding issue. 

This is particularly true of technology which purports to improve 
operational characteristics and thus reduce labor and material costs 
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but does not directly result in revenue generation. The emerging 
technology of PACS fits this definition and is dedicated to 
managing the staggering amount of information which is associated 
with a fully digital image and information support system. Data 
management archive and communication requirements for image 
generation and retrieval will likely exceed 10 gigabytes of 
information per day in the all digital department in a 400+ bed 
institution. 

Costs for such complete systems range in estimates from several 
million dollars to ten to fifteen million dollars depending on the 
technology employed, radiology information system required, 
number of workstations and communication and archiving 
performance specifications. Obviously, an investment of this 
magnitude requires a thorough and detailed cost analysis of 
anticipated cost savings for present and projected operations and 
labor and capital equipment costs associated with the proposed 
technology. 

Although we know of no retrospective studies of the cost/benefits 
of complete PACS installations, the studies for prospective 
application of PACS have been carried out by many investigators. 
Various models/scenarios for these were analyzed by Arenson of 
Penn University (1), Andriesen of Utrecht University (2), Dwyer of 
Kansas University (3), Huang of UCLA (4). They address mostly 
film use and management issues and the possible replacement of 
manual film handling by filmless electronic PACS. 

TMG has developed a comprehensive Computer Model (5) to 
analyze in detail the day-to-day costs of both manual film handling 
and information handling processes (Phase I of the Model) as a 
basis of assessing current and projected costs using presently in 
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place processes. Such analyses have been performed in over 40 
hospitals in the U.S. and Canada and the experience with the TMG 
Model was published (6). Subsequently, the outputs from the 
Phase I of the Model were compared against Phase II of the Model 
which represent radiology film/information management operations 
with PACS. This was done for UCLA and UMD Hospitals and the 
results were presented by TMG (7). From these experiences, we 
concluded that a substantial component of the total cost of manual 
image and information management represents "paper pushing" 
(appointments, scheduling, tracking and results, transcription) tasks 
in radiology. These tasks can be facilitated or replaced by a 
Spectrum RIM system and the time and cost savings from such 
replacement can be substantial. In this presentation we describe 
the application of computer modeling methodology, similar to that 
used previously (7) (8) by TMG for manual film management and 
electronic PACS cost analysis, to perform a cost impact analysis for 
planned installations of the spectrum RIM system. 

Method 

In order to specifically address the cost benefit issues of planned 
Spectrum RIM installations, our previously described (5), (6), (7) 
TMG computerized model for the cost analysis of image 
management in radiology was expanded. This was done in order 
to add procedural steps and their cost element which were 
specifically related to patient information and department 
management. The expansion of the model was done in 
cooperation with the working group at IBM Medical Systems 
specializing in medical/radiology applications. The additional 
elements to the Model and their respective cost/time contributions 
were presented to TMG by an IBM committee of 10 persons which 
encompassed members with experience in diagnostic radiology, 
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radiology administration, radiology nursing, radiology technology, 
radiology information management and medical systems software 
development and applications. Subsequently, the expanded Cost 
Analysis Model was programmed and tested in three medical 
centers and the feedback received from these centers was used to 
refine the Model As a result of the cooperation and feedback 
from these hospitals, the refined Model was finalized and 
subsequently applied for cost impact analysis of proposed 
installations of Spectrum RIM systems. 

Results 

The manual vs. RIM costs were analyzed in the above mentioned 
hospitals in relation to the bed size and annual examination volume 
in respective radiology departments. The summary analyses of 
technical operating cost component for these hospitals are shown 
in Table 1 below. 



Hospital 
Symbol 


Bed 

Size 


Annual 
Exams in 
Radiology 


Annual Tech Cost 
with Manual Info 
Mgmt. 


Annual Tech 
Cost with RIM 


Difference 

% 


1 


630 


148,000 


1,892,000 


1,781,000 


5.9 


2 


577 


83,000 


1,668,000 


1,430,000 


14.2 


3 


334 


59,000 


814,000 


672,000 


17.4 



Table 1: The comparison of annual technical operating costs 
for manual vs. Spectrum RIM information 
management method in the three tested hospitals. 
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The technical costs illustrated above represent actual hospital 
spending allocated to labor and materials for patient image 
handling and information management. If the RIM installation is 
to bring about anticipated savings, the methods and means of 
patient information management and the work flow in the 
department should be readjusted. This may require re-qualifying 
and/or reviewing some of the personnel positions. If and only 
when these changes are implemented, the RIM system can bring 
about anticipated effects. In addition to the above day-to-day 
potential savings and other benefit of RIM installation may be in 
the cost avoidance category. It is a general trend that the annual 
procedure volume in radiology increases from 3 to 15 percent. 

Such increases are usually associated with additional costs of hiring 
new personnel and spending funds on materials and other costs. 
If the RIM installation can facilitate work flow and the time 
required for tasks associated with "paper pushing", the costs of 
hiring additional help to accommodate increased volume can be 
avoided altogether. The Table 2, below, illustrates the comparison 
of manual vs. RIM operation for 10 year projected cumulative 
labor costs. 
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flOSpiIal 

Symbol 


10 yr proj. labor 
(manual) (PV cumulative 
in SlOOO's) 


10 yr proj. labor 
(RIM) (PV cumulative 
in SlOOO's) 


Cumulative Cost 
Avoidance (PV in 
SlOOO's) 


Tech 


Cler 


Tech 


Cler 


Tech 


Cler 


1 


9,510 


4,155 


6,906 


2,386 


2,604 


1,769 


2 


4,267 


3,986 


3,222 


1,925 


1,045 


2,061 


3 


2,969 


1,941 


2,234 


924 


735 


1,017 



Table 2: The comparison of cumulative labor costs 10 year 
projections for manual vs. spectrum RIM information 
management method in the three tested hospitals. 



In discussion with the institutions who used the model for 
evaluation of the Spectrum RIM installation, they had the following 
comments regarding the model itself and the application of 
modelling to cost justification issues: 

■ The TMG model methodology is scientific and thorough. 
The model can be customized to each institution which is 
perceived as a requisite because of the differences between 
institutions. 

■ The data provided by the model is helpful as a base and 
indicator of cost justification issues in consideration of 
acquisition of a Spectrum RIM system and the concept of 
modelling is useful. However, modelling and cost 
justification are not adequate by themselves to justify the 
acquisition; there have to be other reasons, e.g., improved 
performance, quality of care, etc. 
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The model identifies increments of time and costs that can 
hypothetically be saved by installation of the Spectrum RIM 
and the assumptions underlying the hypothesis can be 
changed for "what if analyses. However, a question is 
whether all the aggregate savings will, in practice, add up to 
higher throughput or lower PTE's. The issue is, assuming 
the savings be made, whether the combination of savings 
will work out in a way that is operationally acceptable and 
whether the time saved can be put to good use. 

The modelling does help in identifying where savings can be 
made through operational improvements and acquisition of 
technology. It is a good complement to the acquisition 
process. 

It will be helpful when adequate data has been accumulated 
to compare actual experience to hypothetical savings. 

There is a fair amount of time involved in collecting the 
data but it's recognized as necessary to obtaining an 
accurate analyses. "You get out of it what you put into it," 
was the comment of one participant. 

The report is "a little overwhelming and busy when you first 
look at it but when you review it, all the information is 
there." 
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Summary 

In summary, the hospital receives two management reports with a 
Executive Summary supported by a detailed spread sheet analyses 
of procedures, task times, consumables and amortized capital 
equipment expenditures. 

Phase I portrays costs of current methods of information and 
management and highlights such areas as costs by modality and 
exam, professional and technical costs, comparison to national 
averages, with reference staffing and procedure volume, 10 year 
projected costs if no action is taken and an evaluation of the cost 
savings potential of reduced hospital stay. 

Phase II presents the projected costs by installing the Spectrum 
RIM and highlights comparative areas of savings over a 10 year 
period of costs by modality, technical costs and staffing 
requirements as defined in Phase I. The model also projects net 
cash flow requirements and assumptions on which the model is 
calculated. Following is a graph of comparative professional 
staffing levels and technical costs taken from the Executive 
Summary of Phase II for one of the hospitals participating in the 
clinical evaluation program. 
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No Radiology Information System currently 
on the market is perfect, nor can any one prod- 
uct be expected to be all things to all institu- 
ions. Although the better programs, for the 
most part, provide the needed functionality, the 
implementations are quite different and, at 
times, more closely resemble a computer engi- 
neer's concept of an X-ray department than actual 
practice. Because of their size and complexity, 
these programs are relatively inflexible and are 
difficult to change to suit departmental require- 
ments. The "tailoring" that is allowed for 
newly purchased systems tends to be simply cos- 
metic in nature. Furthermore, the large firms 
that produce these programs, such as IBM, DEC, 
etc. are loathe to admit faults in program de- 
sign and even once they do, it is a matter of 
months or years before the corrections are in- 
corporated in the next upgrade. 

With proper networking, however, it is 
possible to correct many of these awkward or 
faulty processes and actually extend the capa- 
bilities of the program by offloading functions 
from the central computer and then altering them 
as needed. For these purposes, data is sent 
from the host computer, which now acts as a file 
server, to one or more PCs for further manip- 
ulation and, if desired, independent output. 
Usually, only minimal programming of the PC is 
required because of the availability of a wealth 
of reasonably priced software that can provide 
almost any desired function. The inviolable 
rule for these procedures, however, is that 



47 



48 



whatever is done cannot affect the code of the 
management system or alter its database* If 
this is not adhered to, maintenance of the 
management system becomes almost impossible and 
software support contracts are voided. This can 
be accomplished, with DECrad, as it uses 
standard VMS routines for some of its processing. 
Data acquired at this point is completely out- 
side of the management program. 

The backbone of our information system is 
an Ethernet cable running DECnet * The manage- 
ment system runs on a VAX 785, with a VAX 780 
normally used for image processing research, 
that is also on the network available as a re- 
dundant system. Multiple Macintosh computers 
are accessible across the same network. Each is 
equipped with an Ethernet controller board and 
runs TSSNet, a true DECnet emulation including 
both the File Access Language and the DECnet 
mailer. The Alisashare sytem allows us to use 
an RA90 disk drive on the VAX as storage for 
the PCs. With these networking tools, it is 
possible to exchange data and mail between the 
VAXen and Macintoshes in both directions. 

The administrative section of the depart- 
ment runs on IBM PC systems and clones. A 
VAXStation 3100 is connected as a single node 
on the Ethernet cable, utilizing DEC f s PC LAN/ 
server software. The PCs are daisy chained to 
each other with thin wire Ethernet, which is 
then connected via a DESPR, a local repeater, 
to our backbone cable. The PCs run DOS DECnet 
and have the capability of exchanging mail be- 
tween themselves as well as with any node on 
the network. 

Enhancements 

Once film reports and management data are 
archived on tape, they are difficult to retrieve 
in a form that is easily manipulated. This 
limitation applies to most of the commercially 
available management systems. Using a small 
filter program on the VAX, we are able to 
export demographic and administrative data from 
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the daily billing tape to a relational database 
on the Macintosh. This is now available for 
sorting, searching or restructuring as desired. 

Archived radiology reports are also a 
problem. In the process of archiving to tape, 
all formatting characteristics are stripped. 
Thus, when retrieved the reports bear little 
resemblance to our standard radiology reports. 
This has been questioned by referring physicians 
and insurance companies. Furthermore, the 
sequential search required by tape storage is 
not optimal for the purpose. We plan to archive 
all outdated reports with the Macintosh and 
optical disks in order to give us random access 
and to avoid the aesthetic dropout. 

With our management system, it is not 
possible to access more than one online X-ray 
report in an efficient manner. This can be 
corrected by copying the patients X-ray history 
onto the Macintosh, thus providing a list of 
examination ID numbers. Utilizing a split 
screen, any number of individual reports can 
then be called up in random fashion. 

The PCs are also used to maintain individ- 
ual, specialized databases for the various 
sections in the department. The customized 
database programs run on Macintoshes but the 
data files are kept on the VAX because of the 
large available storage space and the conve- 
nience of doing only one backup a night and 
securing all data. We also are using the PCs 
for generation of mammography reports and are 
in the process of extending this capability to 
other fields such as angiocardiography, nuclear 
medicine and, possibly, bone radiology. 

Utilizing the graphical capabilities of 
the Macintosh, we have demonstrated the feasi- 
bility of capturing digital images from the net- 
work and adding them to the printed radiology 
report. It is also possible, if the images are 
read off the Macintosh screen, to add arrows 
and numbers during interpretation, with almost 
no extra effort required of the radiologist, 
and so produce an annotated report. 
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Introduction 

This paper describes the entity-attribute- value model for data and procedure, a model 
which offers some promise for the solution of radiology integration problems. The 
model's merits are discussed in terms of its potential ability to integrate data types 
common in radiology, and to solve three specific integration problems. Examples 
and rough benchmark statistics from the VAX DECrad V4.0 product are reported. 

Radiology Informatics 9 Current Problems 

Need for Integration: The most important challenge currently facing the field 
of radiology informatics is the need for integration. Though specialized radiology 
applications continue to make and require functional improvements, their individual 
sophistication currently far outstrips their aggregate ability to provide a unified envi- 
ronment for the practice of radiology. Progress in this arena is being made, primar- 
ily through the development of standardized interfaces [3], [6] but no dominant inte- 
gration strategy has yet emerged. Despite such efforts, the importance of integration 
continues to increase. Both competitive pressure on hospitals and the increasing 
complexity of specific applications contribute to this trend. The anticipated trend to- 
ward patient-centered hospital integration [10] will result in further pressure not 
only to integrate the radiology department, but also for radiology components and 
systems to adapt to and support the needs of an overall integrated healthcare comput- 
ing environment. 

In addition to the clear need for integration, radiology informatics also needs to pro- 
vide greater flexibility than is now the norm in specific applications such as informa- 
tion systems. In particular, users of radiology computing systems require improved 
abilities to specify the details of their systems' operation. 

In addition to meeting the foregoing needs, any strategy for integration in radiology 
must account for those areas of technology whose ongoing rapid development will 
most significantly affect the field: image storage, retrieval and manipulation capa- 
bilities [9], computer assistance in diagnosis and treatment, automated processing of 
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text [5], and applications of neural networks [3]. 

Generic Needs: At a low level, the requirements described above may be sum- 
marized as the need for an integrated environment which handles disparate data 
types in a homogeneous fashion. The most important of these types encountered in 
radiology are as follows: 

1. image - bitmap representations of medical images, subject to high-speed re- 
call, presentation and manipulation; 

2. discrete - data which fit the traditional database model, wherein data is cate- 
gorized into fields and records, such that every field within a particular re- 
cord has a unique value; 

3. numeric - discrete data upon which calculations may be performed; 

4. text - radiologic findings comprise the most important application of this data 
class, but their use permeates radiological computing, including diagnosis 
and procedure summaries, coded titles, incident reports and numerous other 
examples. 

There are numerous data types encountered in radiology which do not fit clearly into 
these categories, and there are also numerous structures which consist of hybrids of 
more than one basic data category. The low-level integration of these basic data 
types is a prerequisite for any overall radiology integration strategy. 

Example Problems: At a higher level, integration in radiology consists of a 
number of complex and interrelated problems, not all of which have yet been clearly 
defined or even identified. This paper will consider simple cases of three such prob- 
lems: (1) interfaces between Picture Archiving and Communication Systems (PACS) 
and Radiology Information Systems (RIS), (2) integration of radiological text, nu- 
meric and discrete data, and (3) automated text analysis. The following sections pro- 
pose a low-level data and functional model for radiology integration, and evaluate its 
theoretical merits on the basis of both these example problems and the low-level re- 
quirements stated above. 

The Entity-Attribute-Value Model 

The entity- attribute- value (EAV) model is a method of repj esenting knowledge of 
the types traditionally called data and procedure in a homogeneous form. This form 
consists of a three-level hierarchy containing entities, attributes and values, in that 
order of significance. Data is randomly accessible via indices at each hierarchical 
level 
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Entities are the subjects of information or knowledge stored in an EAV database. 
Attributes are properties by which information about entities is organized. Values 
are the specific data stored. The unique combination of an entity, attribute and 
value ("EAV tuple") represents a fact. For example, the idea represented in English 
by "An apple is red in color" may be represented as the EAV tuple <AP- 
PLEICOLORIRED>. (For the remainder of this paper, angle brackets will delimit 
EAV tuple examples; entities, attributes and values will appear in order, delimited 
by the "I" character). By their hierarchical nature, EAV databases ensure that all 
data concerning a particular subject (entity) are grouped together in a defined order, 
and within a subject, that all values for a given property (attribute) are grouped to- 
gether in similar fashion. Thus, in an EAV database, knowing an entity, one can im- 
mediately retrieve a list of attributes which pertain to that entity. For any entity and 
pertinent attribute, one can similarly retrieve a list of values. An entity-attribute 
combination may have more than one value. For example, the entity APPLE might 
have both values RED and GREEN for attribute COLOR. It is important to note 
that EAV data structures are recursive: i.e., that an element (entity, attribute or 
value) of a tuple may itself be a tuple. 



Figure la: Relational Database Example 
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Traditional databases indi- 
cate the significance (attrib- 
ute) of a data item (value) 
to the record (entity) in 
which it occurs via the 
item's physical location. 
Figure la provides an ex- 
ample of a traditional data- 
base which might be used in 
a radiology application. 


Figure lb: EAV Representation 

<123IDATEI6/5/90> 
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IEXAMICH1> 
<456IDATEI6/10/90> 
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IEXAMIUSTHO> 



EAV databases, on the other hand, represent the relationship between entities and 
values directly via the attributes. The database shown above might be represented 
by the EAV tuples presented in Figure lb. Note that empty "fields" are not stored in 
the EAV representation. The relational model offers an economy over the EAV ex- 
ample in that attributes in the relational model are implied by position, and do not 
need to be represented directly. Use of key compression techniques, and the 
model's ability to avoid storage of placeholders for missing data will enable EAV- 
model databases partially to offset this shortcoming. 
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It is possible to construct variants of the EAV model which retain its essential char- 
acteristics. The prototype EAV implementation associated with VAX DECrad V4.0, 
further discussed in the Appendix, makes use of a four-member EAV tuple. The 
first three members of this tuple correspond exactly to the EAV model as presented 
above. The fourth member is designated as "text" or "comment," corresponds one- 
to-one with an associated value, and is not indexed. This scheme permits efficient 
storage of large atomic data structures such as text blocks. 

Standard databases allow direct access to records (entities) via indexed "key fields." 
Access to records via non-key values requires either searching an entire database 
and filtering out unnecded data, or maintenance of cross-reference indices for any 
non-key fields by which users wish to select data. EAV-model databases are sub- 
ject to the same limitations, but the model permits cross-reference indices and other 
standard (and nonstandard) database constructs to be integrated into the same homo- 
geneous architecture as the main database, with resulting gains in simplicity, uni- 
formity and flexibility. The mechanics whereby this integration is accomplished 
are presented in the following sections. 

Applications to Radiology Computing 

Low Level Constructs: The EAV model is amenable to the development of a 
wide range of tools necessary for an integrated radiology environment. It is produc- 
tive to consider these applications first at the lowest level, then consider how low- 
level EAV tools might be used to solve practical problems of radiological integra- 
tion. 

The most basic task within an integrated radiology environment will be to provide 
those services currently performed by traditional database technology, The preced- 
ing section has explained how a traditional database (using a relational example) 
might be implemented under an EAV model, but has not treated the implementa- 
tion of such ancillary data structures as inverted keys. Implementation of these 
structures is relatively straightforward under the EAV model. Cross-references may 
be implemented by assigning a unique cross-reference entity to every attribute to be 
inverted. For every instance of that attribute in an EAV tuple, a corresponding tuple 
consisting respectively of the cross-reference entity, the original value and original 
entity is stored. This procedure enables rapid retrieval of lists of entities which ex- 
hibit a particular attribute and value. 

Consider the example attribute COLOR, and suppose that it has been assigned the 
cross-reference entity COLOR-XREF. Then the tuple <APPLEICOLORIRED> will 
be inverted as <COLOR-XREFlAPPLEIRED>. Traversal of the subtree <COLOR- 
XREFIRED... will retrieve APPLE, as well as all other entities red in color. Tra- 
versal of the subtree <COLOR-XREFL. will retrieve a list of all colors which have 
been used to describe entities in the database. 
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The implementation of list structure, a fundamental building block of classical arti- 
ficial intelligence (AI) [1] has been presented implicitly in the preceding example. 
Lists are simply EAV tuples with one or more elements unbound. For example, 
<COLOR-XREFIPURPLE I... (entity and attribute bound, value unbound) represents 
the list of all things purple in colon A use of list structure applicable to the current 
example would be to reserve a special entity to contain lists of attributes and their 
cross references, e.g.: <XREF-USTICOLORICOLOR-XREF> 

ISHAPEISHAPE-XREF> 
ITASTE1TASTE-XREF>... Such a list 
will enable an application to determine an attribute's cross-reference entity rapidly. 

The preceding example also implicitly presents another important property of EAV 
databases: that cross-references are combinations of method and data structure. 
EAV databases differ from their traditional counterparts in that the methods by 
which cross-references are created and maintained may be represented within the 
same homogeneous structure which contains both database and cross-references. 
This integration is achieved by the implementation of rule processing within the 
EAV software. 

Rule processing within the EAV model may be achieved by developing a method 
for identifying certain classes of EAV tuples as rule hypotheses, and assigning to 
each hypothesis tuple one or more "conclusion" tuples, whose assertion (storage) is 
to be caused by the assertion of the hypothesis. This method has been applied with 
some success to VAX DECrad V4.0's Case Indexing functions, as described in the 
Appendix. Under this method, a specific entity is reserved for storage of rule infor- 
mation. Under this entity, the attributes consist of hypothesis tuples, and the values 
of conclusion tuples. For optimum performance, hypothesis recognition depends 
most significantly upon a constant element of the tuple (attribute in DECrad *s case), 
then optionally on another element (value in DECrad). Hypothesis tuples are reor- 
ganized so that their recognition elements occur first, to enable rapid retrieval by the 
recognition elements. This implementation of rule processing also requires a means 
for representing unbound elements in both hypothesis and conclusion tuples. The 
convention used in DECrad' s EAV implementation (and for the remainder of this 
paper) represents universal quantification ("for all" or "for any") with square brack- 
ets ("[]") and definite quantification (replacement of a previously unbound value) 
with braces ("{)"). Under this scheme, <RULEi<srzEiBiGiiX]>k{X ) iweightikeavy» 
represents the rule expressed in English as "for any X, 'X is large in size' implies 
*X is heavy/" Clearly, additional rule -processing tools such as Boolean and proba- 
bilistic operations on hypotheses, and more sophisticated binding techniques are re- 
quired for applications which are to approximate reasoning, but even the simple 
tools described are sufficient to represent effectively the methods necessary for da- 
tabase cross-referencing. For example, the rule 
<RULEl<COLORIREDI [X]>kCOLOR-XREFlREDl { X } » expresses the cross- 
reference method exemplified earlier. Since EAV tuples are recursive, methods 
can be generalized: 
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<RUl^kNATL^IATTRlBUTEJfX]>kRUXB<(X)l[A]l[B]>k{X} for exam- 

ple, represents a general rule for creating cross-reference entities and rules. 

Another important aspect of EAV structures is their ability to represent algorithms, 
both in traditional forms, and in forms such as goal trees favored by artificial intelli- 
gence applications [1]. Traditional algorithms may be represented in simple sequen- 
tial manner, by using values to indicate both step sequence and operations to be per- 
formed. Alternately, hierarchical modular structure may be expressed by nesting of 
tuples or by recursion in which values or portions thereof recursively indicate other 
entities ("value-entity recursion"). Goal trees are also easily represented using 
value-entity recursion. Such algorithm representations require two features in order 
to be used by application software: links from elements in the algorithm representa- 
tion which denote actions to the software objects which perform the actions, and a 
method of invocation for algorithms so represented Possibly the simplest invoca- 
tion method involves assertion of an EAV tuple containing the algorithm representa- 
tion. Under this method, a linguistic means must be provided to suppress the stor- 
age of asserted tuples; otherwise, flow of algorithms may result in spurious effects 
in EAV database structures. 

This type of recursion may also be used in constructing augmented transition net- 
works, which are important tools used by applications attempting to parse natural 
language texts [1]. Chamiak [1] defines such networks (ATNs) as transition net- 
works (directed graphs whose arcs are designated as tests of words for their parts of 
speech [1]) with three adjuncts: (a) a data structure containing the current parsing 
history; (b) a structure specifying atomic operations on sentence constituents (e.g., 
converting singular to plural); and (c) structures specifying methods for creating, in- 
serting an deleting constituents. Directed graphs with labeled arcs may be con- 
structed relatively simply using value-entity recursion. Structure (a) above may be 
implemented from traditional database and list components, and (b) and (c) as goal 
trees. Thus an entire ATN may be represented within the context of an EAV struc- 
ture, 

EAV In Radiological Examples: The previous section has described how 
several low-level constructs of great importance in AI and traditional applications 
may be constructed using the EAV model This section will describe how these 
constructs may be applied to solve the example radiology integration problems pre- 
sented in the introduction. 

Integration of disparate data types : The first such example concerns the integration 
of functions currently performed by a traditional R1S. Most commercially available 
RISs currently use traditional database technology to store their data. Such systems 
suffer from the drawback that retrieving database subsets requires passing through 
an entire database, restricting by a limited number of inversion keys, or custom pro- 
gramming. It is relatively easy to construct a software interface which equates die 
traditional database representation of data used by an RIS with an EAV representa- 



56 



lion, and automatically calls a generic EAV assertion routine to "echo" storage of 
traditional RIS data in the EAV structure. A simple interface of this type has been 
included in VAX DECrad V4.0. This interface echoes the storage of a subset of the 
RIS database (contents of the subset are specifiable by system managers) in an EAV 
structure. All attributes corresponding to fields defined in the RIS dictionary have 
cross-reference entities defined by a generic rule of the form given in the previous 
section. Text blocks stored in the conventional RIS are parsed into individual 
words, which are asserted as values for specialized text attributes. At present, the 
only database-correspondent entities in the EAV structure are radiology examina- 
tion identifiers (accession numbers), although the system is easily extensible to in- 
clude other entity types such as patients, personnel, resources, and others. The ho- 
mogeneous inversion structure enables users to retrieve data based upon extremely 
flexible combinations of attribute/value criteria. These structures also make optimi- 
zation of such inquiries almost trivial. For example, development of a query to find 
all cases with diagnostic reports containing one of a number of alternative phrases, 
for patients bom within a specific year, exhibiting one of several complaints, is a 
simple process, requiring no programming, of constructing and evaluating simple, 
verifiable subquerics, and assembling them into the completed query. Another sim- 
ple interface permits RIS users to perform a subset of RIS functions upon retrieved 
lists of entities. 

PACS-RIS interfaces; The second example problem presented in the introduction 
involves the specification of a PACS-RIS interface. A number of vendors and insti- 
tutions have devoted and continue to devote enormous efforts to this problem [7], 
[6], but no definite solution has yet emerged. The Marburg Model, [7] upon which 
much of this work is based, postulates that PACS-RIS interfaces may be achieved 
by analyzing an entire radiology operation into component functions, and specifying 
the interfaces between these functions. Some of these interfaces are currently per- 
formed by RIS, others by PACS, and others, as yet undefined, will constitute the 
interface. Greinacher et al. use the Marburg Model to build a unified data model of 
radiology based upon entities and their relationship to one another[6]. They then 
continue by specifying typical entities and attributes encountered in radiology. The 
entities so specified are the fundamental conceptual units upon which interfaces 
may be built EAV constructs are suitable for representing both the overall and de- 
tailed models developed by Greinacher et al Their detailed model is expressed in 
EAV terms in its definition. The overall model, represented as a directed graph 
connecting entities and relations, may be represented quite simply in EAV structure 
using value-entity recursion. The interfaces themselves may communicate data us- 
ing a nested entity -attribute- value format. Under such a scheme, a transaction might 
contain a single entity, then one or more attribute-value pairs (called "name-value 
pairs" in other literature, e.g. [3]) to specify data pertinent to the indicated entity. 
For economy, any attribute for which multiple values were transmitted might be 
transmitted only with the initial value, and its repetition for the remaining values 
could then be implied by space-saving device. Such interfaces may be gaining ac- 
ceptance in the general field of medical informatics [3]. Another way in which the 
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EAV model might benefit such interfaces is by increasing their flexibility. Current 
interfaces, both internal and external to radiology, tend to feature event- triggered in- 
formation transactions [3]. The triggering mechanisms which associate events in 
host systems with the transactions they cause are typically hard-coded in the soft- 
ware of the host systems. This situation leads to extreme inflexibility of interfaces: 
difficulty of customization, and diversion of software development resources to en- 
gineering and support of hard-coded interfaces. One approach to this problem in- 
volves recognizing event- triggered transactions as a special case of query output. In 
such an approach, queries must be able to specify temporal and other conditions un- 
der which the query is to be evaluated, as well as information constraints currently 
supported by most query software. A typical query under a system currently in op- 
eration might be rendered as "retrieve and transmit patient names, responsible physi- 
cians and diagnoses for all chest studies performed within the last 36 hours/ 1 A 
query capable of specifying trigger events might be rendered as "From now until 
further notice, whenever a chest exam is performed, retrieve and transmit patient 
name, responsible physician and diagnosis." By virtue of the homogeneity with 
which it represents data, the EAV model is suitable for expressing such queries. By 
virtue of its ability to represent retrieval methods (goal trees) and discrete proce- 
dures (traditional algorithms), it is also suitable for evaluating them. 

Automated text content extraction : The third example problem posed in the intro- 
duction concerns automated parsing and extraction of semantic content from radio- 
logic texts, and integrating such information with other radiological data. Promising 
work in this area is currently being conducted and reported by Gabrieli and Speth, 
among others [5]. Their work involves processing medical texts, and representing 
facts contained in them in an hierarchical coding system. It is expected that the im- 
portance of such sub- applications will increase dramatically in the near future [10]. 
The EAV model will prove quite useful in this area as well as those described previ- 
ously, due to several factors: (1) the ease with which it may be used to represent 
ATNs and other constructs used in natural language processing; (2) its ability to rep- 
resent knowledge in a simple, intuitive, recursive form, which will lend itself to rep- 
resentation of semantic content; (3) its facility for storing hierarchical fact codifica- 
tions of the type used by Gabrieli and Speth; and (4) its ability to integrate these fea- 
tures with traditional methodologies and structures as well as functions yet to be 
developed. 

Impact of Emerging Technology: In evaluating the viability of any pro- 
posed model, it is necessary to assess its ability to react favorably to foreseeable de- 
velopments in its surrounding environment. An extremely significant trend in mod- 
em computing is the development of massively parallel hardware architectures and 
the associated evolution of software concepts. One such concept which is rapidly 
gaining importance is the neural network or "connection model,' 1 Neural networks 
have recently received considerable attention as computing structures, and appear to 
offer considerable benefits to the solution of a number of problems including visual 
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Figure 2a: A Neural Network 




Figure 2b: Matrix Form 

To: 



pattern recognition and learning [4], and solution 
of complex diagnostic problems [8]. Based 
upon theoretical models of the human central 
nervous system, they specify computational 
structures as networks of simple nodes which 
operate at various states of "excitement/ 1 The 
activity of any node in such a network may po- 
tentially either excite or inhibit the activity of 
other nodes. Such networks may be represented 
as directed graphs with weighted arcs as exem- 
plified in Figure 2a, Current neural theory finds 
it useful to represent such networks in matrix 
form. The matrix representation of the example 
network is shown in Figure 2b. Matrix repre- 
sentations of such networks are useful for simu- 
lating learning and response to stimuli via ma- 
trix multiplication [4]. Representation and ma- 
nipulation of neural networks in EAV structures 
is again relatively simple. The neural net de- 
picted in figures 2a and 2b may be represented 
by the EAV tuples <AIBI-10>, <A1CI+3>, 
<BIAI+7>, <CiAI+5>, <C!B]-6>, <DIA1-6>, and 
<DIBI+4>. Again, the EAV model's ability to avoid storage of placeholders for 
missing data give it a decided economic advantage over the matrix model, especially 
in large-scale cases which are especially sparse (cases with large numbers of nodes, 
where each node is connected to relatively few other nodes), which may account for 
a large portion of neural net applications. This economy will apply to processing 
times in such cases as well as storage requirements; assuming a system of m nodes, 
each connected, on average, to c other nodes, the matrix model will require n? multi- 
plication operations to complete a learning or recognition event, while the EAV 
model will require mc". D'Autrechy et ah have used a structure quite similar to 
EAV with advantage to represent neural networks in the MIRRORS/TI project [2]. 
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Summary 



The entity-attribule-value model displays coasiderable promise for solution of inte- 
gration problems in radiology informatics. This potential is the effect of several use- 
ful properties; (1) the model's ability to accommodate traditional database methods, 
low-level AI structures, and control structures; (2) the ease with which it may be in- 
tegrated with existing applications; (3) its adaptability to such emerging computing 
technologies as neural networks, and (4) the simplicity, uniformity and flexibility 
which result from die homogeneous implementation of all of these elements. 
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In order to achieve the level of integration required by the increasingly sophisticated 
radiology environment, the field of radiology informatics will need to pursue unified 
architectures and data models such as EAV. Although this model is potentially use- 
ful in its own right, it is presented here to inspire thought and discussion of such 
models and architectures in general. 

Appendix: EAV in VAX DECrad V4.0 

The development of the Case Indexing features of Digital Equipment Corporation's 
VAX DECrad V4.0 product has provided a productive test of some of the concepts 
discussed in this paper. VAX DECrad is implemented in the Digital Standard 
MUMPS (VAX DSM) language and database environment. The Case Indexing fea- 
tures offer radiology personnel the opportunity to retrieve exams rapidly from the 
DECrad database according to flexible criteria on homogeneous attributes, and to 
perform selected DECrad functions on the retrieved exams. The primary users of 
these features are expected to be radiologists developing interesting case files, ad- 
ministrators tracking service quality, and film librarians. A number of facts con- 
cerning the development of these features are summarized in the following list: 

1. The phases of the implementation required roughly the following allocations 
of FTE personnel: general design, 12 days; prototype implementation, 25 
days; design of revisions and extensions, 3 days; implementation of revi- 
sions, 45 days; quality assurance and associated revisions, 25 days; technical 
documentation, 12 days; user documentation, 40 days (total 152 days). 

2. The completed prototype system comprised 60 DSM routines, 14 interactive 
screens, 7 fixed-format reports, and a rule-base of 30 rules (as supplied with 
default DECrad V4.0; the rule-base is user-extensible). 

3. Benchmarks on a development system indicated that activation of all possi- 
ble EAV options at a large (1000 diagnostic reports per day) radiology site 
would consume approximately 1% of one VAX Unit of Processing (VUP) in 
CPU resources. This benchmark was based upon an observed EAV- 
assertion throughput rate of 320 bytes per second on a VAX 11/780, and an 
average estimated EAV assertion volume per exam of 1900 bytes (tests con- 
ducted at the University of Massachusetts Medical Center found mean diag- 
nostic report sizes of 580 (SD 520) bytes). 
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Development of a Product Inventory System for a 
Diagnostic Imaging Department 

Alicia Giangrande and Michael Taricani 
Yale-New Haven Hospital, 
New Haven, CT 



The delivery of diagnostic and therapeutic imaging 
services demands that a medical product inventory be 
readily available. For our department over 80V. of product 
inventory occurs in Special Procedures which consists of 
a Cardiac Catheterization Laboratory and a Peripheral 
Vascular Interventional Laboratory. Between the two 
laboratories about 1200 different items are used with an 
inventory of about 5000. Inventory management and the 
ordering process can be a time-consuming task. In this 
paper, we would like to share with you our experience in 
developing an inventory control computer system to help 
manage product ordering. 

Our inventory system had three main objectives. We 
wanted the system to keep track of shelf numbers and 
print a daily list of items to order. With a system that 
stores item usage, we also wanted to reduce on-shelf 
inventory especially underutilized or unpopular items. 
The system, which was to duplicate our manual card- 
requisition system, also was to provide statistical data 
concerning number of items used, inventory value and 
budgetary projections. Our second objective was to 
choose a database that could easily handle changes and 
was relatively inexpensive. Our third objective was to 
create a system that was easy to use so that 
technologists could continue their responsiblity of 
product ordering. Having had experience with Q&A and 
knowing it was easy to use and inexpensive, we decided to 
choose it as our database software. 
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Our first task was to develop a coding scheme for each 
inventory item. The codes needed to be easily recognized 
within a 10 character limit. The codes in the inventory 
system correspond to the codes used for billing in our 
Radiology Information System (RIS). The first character 
identifies each distinct budgetary department. In this 
case S is used for Specials. The second character 
identifies the subdepartment which is either a C for 
Cardiac or a P for Peripheral . The f ol lowing characters in 
both the Cardiac and Peripheral groups further identifies 
the product by specifying the technique for which it is 
used and/or the size, product type and style. As an 
example, a cardiac adult angioplasty balloon of 1.5 f rench 
size and "Skinny" style is identified as SCAAB1 . 5SK . By 
logically grouping products we are able to retrieve a 
product record by using a partial lookup. For example, we 
can retrieve all product records which are adult 
angioplasty balloons of 1.5 size. In addition, a sort by 
codes categorizes records so that comparisons can more 
readily be made. In the end, we had 1200 items coded with 
40X of those items from the Cardiac section and 607. from 
the Peripheral group. 

Next, we developed the system's screen design. This has 
undergone several changes. Figure 1 shows our most 
recent design. The initial screen is divided into three 
parts. The first part contains user and product 
information. As a security precaution, a user must enter 
their initials along with their password to enter the 
system. The record or "card" for each inventory product 
keeps track of the current and previous user. In this 
way, if entered data is in question we can better resolve 
the issue. The product information includes the product 
code which uniquely identifies each item as well as its 
group (SCAAB. . , SPT. . , SCACG. . ) , catalog number, description, 
price per item, vendor, date of inventory startup, startup 
inventory number as well as a Y/N field for product 
specifications. The starting inventory number is used 
later in the "Summary of Product Use". The product 
specification field is a cursor control field which allows 
the user to skip to the bottom of the last screen so that 
one can view product specifications. 
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The next part of the screen shows the total number of 
items used, ordered and received as well as the number of 
items stil 1 on order. These are calculated fields. Items 
used are updated on a daily basis from a flat file which is 
downloaded from the RIS via Kermit into the PC. A 
download contains those items charged in the RIS the 
previous day with Monday containing items charged Friday 
as wel 1 as the weekend . The download file is then imported 
into Q&A and the individual product records are updated 
by matching product codes. The Total Ordered and Total 
Received fields are the addition of individual ordered and 
received entries which are subsequent system screens. 
(Please see Figure 1) . We believe that the most heavily 
used product will need at most about twenty-five order 
and receive slots. Thus in the databases for Cardiac and 
Peripheral we hope to store one year of inventory data. 
The field out-on-order is calculated by subtracting the 
Total Received from the Total Ordered. In theory, all that 
is ordered under a Purchase Order (PO) number should be 
received. The YTD cost is the Item Price multiplied by the 
Total Used. 

The last part of the screen is a "Summary of Product Use" . 
The Shelf Stock describes the number of items on the 
shelf. This is calculated as Total Received minus Total 
Used plus the Starting Inventory Number. The Received- 
Used-Par field is a relationship that describes the 
difference between received and used in relation to the 
user-defined par level . The next field describes the 
number of items to order. It is the addition of the 
previously described relationship number (Received-Used- 
Par) with the starting inventory number (Start Inv) and 
the amount out-on-order (Out on Order). The amount to 
order is the opposite of the result of this calculation. 

The system works well. All pertinent information 
concerning a product appears on the first screen. There 
is a sufficient user audit trail and a security feature 
that prompts a user for a password. In addition, we have 
an error log that records any codes from RIS that are not 
found in the PC database. In this way we can keep the PC 
and RIS codes in synchronization- Examples of inventory 
reports are shown in Figures 2 and 3. Figure 2 shows the 
TO ORDER report which prints after a download occurs. A 
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technologist reviews this list and places necessary 
orders. Figure 3 is a sample management report that 
shows utilization and YTD costs. 

An issue we have encountered with the system has been 
human error in recording items used during a procedure 
and items that are received and/or ordered. In all three 
circumstances either forgetting to enter the information, 
using an incorrect code, miscounting numbers received or 
ordered, and/or mistakenly altering data will affect 
system inventory levels. Overall the computer system has 
been very helpful and has taught us about our inventory 
and its usage. 

In conclusion, we have a few recommendations and we would 
like to share with you a few issues that we are currently 
exploring. We recommend that inventory especially of 
frequently used items should be periodically counted and 
the system updated to account for discrepancies. In 
addition , we favor manufacturer printing of barcode labels 
on products. It is hoped that coding for these labels will 
show some consistency and logic between manufacturers. 
Barcode wanding can reduce key punching; however, there 
is the issue of getting the correct label on a product and, 
once a label is printed, finding a suitable loction to stick 
it. This last issue is especially a concern with smaller 
individually packed items. 

There are two issues we are currently exploring. We 
believe that since we have an appropriate number of items 
on the shelf and we are comfortable with our par level 
settings, we believe a weekly versus a daily download will 
be sufficient. The second issue is to have the system 
provide some notification of increased usages so that we 
can correspondingly increase a product's par level. In 
this way, we can better control our on -she If item numbers. 
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Figure 1. Inventory Screen Design. 
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Figure 3. A sample management report. 



Rearrangement Management Information Facilities (RMIF) 

RJ.S. Boon 

Academic Hospital Maastrich, THE NETHERLANDS 



Summary 

In the Netherlands the focus on Management Information 
Systems, within the (academic) hospital environment, has 
strongly been intensified. This has led to several new 
initiatives, from which the here described one is from 
the Academic Hospital in Maastricht: a joined project 
together with Digital Equipment Corporation. 

This initiative is based on the principle that data, which 
has already been collected on a routine base, could serve 
as feeder(s) for the wanted necessary Management Infor- 
mation. 

Introduction 

The Academic Hospital in Maastricht (AZM) has worked four 
years already with the hospital system of BAZIS. BAZIS is 
a fully integrated HIS that was made to support the oper- 
ational processes like for example: general ledger, 
laboratory research, patient care and nursing. 

The BAZIS system does not provide so far any adequate 
Management Information tool. Due to the process-character 
of BAZIS it is extremely difficult and labor-intensive to 
collect this information, and select or aggregate it. 

Within AZM, together with Digital Equipment Corporation, a 
project has been formulated and started with the name RMIF 
(Rearranging Management Information Facilities) that must 
develop a MIS for middle- and top-management. This new 
system should provide in a quick and easy way answers to 
management questions and should also make it possible for 
them to calculate alternative strategies. 
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In the first project phase (which was finished last autumn) 
the decision was made to start with a separate computer- 
system (from BAZIS) , which has to be loaded with data from 
the BAZIS-system and equipped with a so-called 4GL-tool. 
With this, an end-user could get his wanted information at 
any time of the day without heavy load on the BAZIS-system. 

Today y s situation 

The biggest problem with today f s MIS is that the wanted 
information is not clearly and properly defined and the 
occurrence of a lot of changes during the problem 
definition-phase. Frequently this information is clus- 
tered from different minor database-resources, throughout 
the various process-system parts of BAZIS. These resources 
could contain some million records, so that proposed 
actions could take some hours cpu-time. 

In practice the wanted actions are like: 

- there is a specific need of information 

- this information need should be described carefully 

- as next, it should be investigated from which system- 
resource this information should be withdrawn. This 
happens momentarily with a (third generation) program- 
ming tool with extra features. This work is frequently 
intensive and normally not planned; it runs therefore 
through all daily activities, creating unexpected delays. 
This problem should be resolved of course organizationally 
(which is however not always the case) 

- when everything is tested adequately, it could be re- 
routed to the BAZIS-machine, after which a batchprocess 
could be planned. When this information is needed more 
frequently, there is also a need of written operator- 
guidelines. 

It is quite obvious that this working scheme is troublesome 
and gives unacceptable turnaround- times . 

Solution 

To find a solution for this problem, there has been chosen 
for a (separate) stand-alone computersystem, which looks 
like* ( see Figure 1) 

- LX * e - _ DEC microVAX 3800 with 16Mb internal memory 

- system disk (RA70) with 280 Mb 

- application disk (RA90) with 1,2 Gb 

- tape cartridge (TK70) with 296 Mb 

- tape unit (TSV05) 1600 bpi 

- 7 VT320 displays 

- LN03AH las erpr inter 
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Figure 1. 



This system should be loaded periodically, with the help 
of special developed programs, with data. The system 
should be available the whole day and equipped with 4GL 
tools, so that the end-user could extract his desired 
data himself. 

The software-configuration looks like: (see Figure 2) 

- VMS Operating System (1 - 20 users) 

- VAX/RdB relational database system (including SQL) 

- CDD common data dictionary (backup of metadata) 

- VAX/ RALLY 4GL (application) tool 

- TEAMDATA 4GL (query) tool 

- PASCAL compiler, (RdB precompiler included) 
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Figure 2. 



Further programs have been developed with PASCAL, for 
example; to read data from tape and store it (eventually 
after some specific conversions) into the RdB. 

This, method of providing information, follows the current 
trend, where the user is more and more involved in the 
automation process: at one side he is much more qualified 
and on the other hand the available tools will serve him 
better. 
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Project result 

In the current BAZIS environment a study has been started 
to determine which source-databases are needed to load the 
RMIF-database (see Figure 3) . This loading is being per- 
formed with 3GL tools because frequently conversions are 
necessary and with 4GL this would take far too much time* 
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Figure 3. 



These load- and conversion-programs are written in a 
flexible way so that these could be used for all wanted 
databases. Because source databases in the RMIF sphere 
are huge and numerous, this has led to two problems: 

- with the commonly used 4GL it is impossible to 
aggregate, in an easy way, more databases and 
procedures into a nice overview* (The user is 
far too much involved in basic techniques.) 

- the turn-around time with huge databases is 
unacceptably high. 
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Figure 4. 



The solution for these problems could be found by making 
or setting-up so-called roll-up-structures (see Figure 4). 
These are, to certain views, aggregated databases* 

After this aggregation it is for the end-user then rather 
simple to exploit these databases with 4GL-tools. 

With the help of programs, still to be created, it should 
be possible, within the near future, to develop easily by 
the end-user all needed rollup-structures, and to change 
or modify or to load them himself. 

Conclusion 

It is to be foreseen right now, that with the described 
above, it will be possible, at low costs, with the power 
of a mainframe, but with the ease and comfort of a PC, 
to fulfill the needs of MIS for both top- and middle- 
management . 
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Radiology Information System Interfaces: 
Connecting Your RIS to the Rest of the World 

Robert F. Davis 

Digital Equipment Corporation, Marlboro, MA 



Introduction 

A stand-alone departmental system, such as a Radiology Information 
System (RIS), provides many advantages over software modules in 
an integrated hospital system. However, by definition, a stand-alone 
system is at a disadvantage in its ability to allow access to its data or 
access data from other systems. 

Both the effort of re-entering data already captured and the benefit of 
hospital- wide, immediate access to data as it is entered, makes it nec- 
essary to connect the RIS to other hospital systems. The concept of 
data integration (or sharing data), as opposed to integrated applica- 
tions, has become increasingly important as ancillary systems be- 
come more commonplace in today's healthcare environment. This 
paper discusses experiences with the VAX DECrad radiology system 
and current trends in networking an RIS to other healthcare applica- 
tions. This discussion should benefit others in evaluating, planning 
and implementing links from the RIS to other computer systems. 

Developing DECrad's Interfaces 

The need to get patient demographic data from the Hospital Informa- 
tion System (HIS) became apparent at the first DECrad Version LO 
field test site. This first interface to another system was developed as 
a custom feature for the site and used a simple RS-232 serial line run- 
ning at 300 baud. Data messages were sent to DECrad as patients 
were admitted and discharged from the hospital system. The DECrad 
software read data records from the serial port, identified the individ- 
ual data items in the record and filed the patient demographic data. 
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This basic concept was incorporated into DECrad Version 2.0 as a 
standard feature. The format was changed to fixed length data re- 
cords, and several different types of ADT functions were added. 

During the time that DECrad Version 3.0 was developed, the re- 
quests most often heard were for order entry and results reporting. 
Major enhancements in the design of DECrad' s communications in- 
terfaces, necessary to implement these two functions, were: 

» The ability to designate and run multiple links 

• The use of different transports (RS-232, DECnet, files) 

• The concept of trigger events within the application 

• An event queuing function 

The DECrad Version 3.1 release included a transaction to accept re- 
port dictation status and time from a digitized diction system. 

DECrad Version 4.0 supports the Health Level Seven (HL7) inter- 
face protocol. HL7 is a group of software vendors, users, and consult- 
ants working to standardize the data formats and protocol for com- 
mon transactions. DECrad Version 4.0 also accepts results entry 
transactions, allowing diagnostic reports to be entered from other sys- 
tems. 

The Architecture of Interfaces 

An interface implementation consists of three distinct concerns: 

• The functions that create or are accomplished by data messages 

• The format of the data 

• The physical connection between systems 
Functions 

The most important decision in the design of interfaces, involves the 
functions that the RIS can accommodate, both sending and receiving. 
As the number and type of systems that must be connected increases, 
the flexibility of the RIS interfaces becomes increasingly important. 

It became evident, while designing the results reporting feature for 
DECrad, that the functions within DECrad which trigger the auto- 
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matic transmission of reports, were only a small subset of the many 
key events which occur during the course of a patient visit. There- 
fore, rather than create a specific mechanism for sending reports, DE- 
Crad implemented a generic trigger event system. This included an 
Event Dictionary, which allows the system manager to select which 
events should cause data messages to be sent, and a queueing mecha- 
nism, which "holds 11 the data until the interface is ready to send it. 

The events which define the typical flow of patient data in the radi- 
ology department can be divided into four major categories, and sub- 
divided into several different trigger events. The ability to accept 
transactions which accomplish the RIS functions implied by these 
events, as well as sending data based on each of these events, creates 
a truly open system. 



Patient APT Order Request Perform Study 



Admit 

Transfer 

Discharge 

Register 

Pre- admit 

Merge 



Request 

Approve 

Schedule 

Cancel 

Modify 

Depart 



Dispatch 

Arrive 

Modify 

Begin 

Complete 

Distribute 



Report Results 

Dictate 

Transcribe 

Modify 

Approve 

Amend 



The transmission of charge infonnation for radiology exams is not 
specifically included in this list of events, but is typically associated 
with the complete study or approve report triggers. The content, how- 
ever, must be billing- specific rather than the general event data. 

In addition to these specific events, a general purpose query func- 
tion is necessary. The ability to request infonnation about a patient's 
history, an exam status, or general management information is useful 
to a variety of systems. 

Other events within the hospital, but outside of radiology, could also 
generate a message to the RIS. For example, a physician might re- 
quest a patient's film folder even though the patient is not currently 
scheduled for a radiology procedure. 
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Data Formats 

To understand the messages that are being sent, a system must know 
exactly how the data is formatted. Systems must establish rules for 
transmitting common types of data such as dates, times, and names. 
Also, there must be a way of identifying each individual item of data 
in the message. There are three common ways to accomplish this: 

Fixed Length 

This format requires that each data item occupy a specific number of 
bytes in a specific position within the message. This is easier for 
some systems to decode, but lias the disadvantage of requiring all 
systems to modify their software if the length of a data item changes. 

Variable Leng th 

This format uses some agreed upon character(s) as markers which de- 
limit the individual data items. Each data item is identified by its 
relative position in the message. Incremental changes to length or 
addition of data items can be made without affecting other systems. 

Name- Value Pairs 

This format uses a data identifier code (name) followed by the actual 
data (value). As with the variable length records, changes can be 
made without affecting other systems. It also allows the decoding 
software to be table-driven, making program maintenance easier. 

Physical Connection 

The most common method for connecting systems continues to be a 
point-to-point RS-232 serial line. Many DECrad interface implemen- 
tations use this method of making the hardware connection. It pro- 
vides a reasonable, inexpensive solution for the transaction oriented 
type of interfaces that are necessary. The throughput for this type of 
"network" is relatively low. But, with multiple lines at speeds of 9600 
baud or higher, it should be sufficient for the volume of data gener- 
ated by even large institutions. 

The next evolutionary step toward a more sophisticated network so- 
lution is to use a serial line as the end point connection to a local area 
network (LAN). A temiinal server connected to a DECnet network or 
an interface gateway into a PC LAN creates a good combination of 
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easy access and high-speed, reliable networking, as well as solving 
some of the cabling issues related to RS-232 lines. 

Because these types of connections do not include the network con- 
trol to guarantee error-free transmission, it is necessary to create 
some type of low level protocol that can check for and correct errors. 
In a more robust network environment, this need is eliminated. 

But, creating direct process-to-process communications on a more 
complex network is somewhat more difficult. DECrad accomplishes 
this on a DECnet network to other VAX/VMS systems using a 
psuedo-device, called a mailbox, which appears to the application 
much like a serial port. In a network environment with heterogeneous 
operating systems, or in mixed network environment using gateways \ 
creating the link requires a thorough understanding of both sides of 
the connection and knowledge of what operating system features are 
available. 

However, as open system standards begin to emerge, it should be 
possible for the RIS to understand the details of creating these 
process-to-process connections for the more common networks and 
operating systems, and then offer those as standard options. 

HL7 

The basic concepts of transaction oriented ADT, Order Entry, and 
Results Reporting have been understood for several years, and most 
healthcare systems have some of these capabilities. Implementing the 
actual connections between different vendors has usually been a 
painstaking process. Each application has its own data format, proto- 
col, and functional view. Having two systems agree on and imple- 
ment an interface is often time consuming and expensive. 

In 1987, several software vendors and hospital users met to try and 
address this problem. From this conference, the HL7 Working Group 
was formed and has been working to develop a standard, describing 
this key set of transactions in terms of data formats and protocols. 
Because the focus of the group has been to create a voluntary, ad hoc 
standard, it has been able to move very quickly compared with more 
formal standards efforts. 
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The goal of HL7 is to create an application level standard. The appli- 
cation level is the seventh layer of the International Standards Or- 
ganization's (ISO) network model for Open Systems Interconnection 
(OS I). However, since the ISO network standards do not yet provide 
practical support for most applications, the group has also addressed 
problems dealing with some of the lower levels. 

Conceptually, the standard suggests that real- world events create the 
need for data to flow from one system to another. The trigger events 
cause a message, called an unsolicited update to be sent. The stan- 
dard requires that the receiving system acknowledge the receipt and 
processing of the message. In some cases, such as an order request, 
the acknowledgment may in turn contain some data for the original 
sending system. A slightly different type of message, also defined in 
HL7, is a query. In this case, most of the data flows back to the 
originating system in the form of a response to the query. 

The specification has been designed at three levels to allow a modu- 
lar design in the interfaces. Interfaces written in this modular fashion 
can be easily ported from the least robust network environment (RS- 
232 lines) to a full function ISO/OSI environment. 

The top level is the abstract definition, which specifies the trigger 
events, the data fields sent in a message, and the response message. 
The data is grouped into logical segments and the message is com- 
posed of a message header segment followed by data segments. 

The abstract definition includes, for each data item: 

• a name for the data 

• the maximum length 

• the data type 

• required or optional 

• does it repeat 

This level corresponds to layer 7 of the OSI model. 

The second level of definition addresses how the data is presented, 
and corresponds more to layer 6 of the OSI. The encoding rules 
specify variable length data fields, and use a separator character to 
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delineate the individual fields. All data is represented as displayable 
ASCII characters. The encoding rules also define formats for specific 
data types such as date, time, name, etc. 

The lowest level, appropriately named the Lower Level Protocol 
(LLP), allows HL7 to be used in network environments that do not 
provide basic transport reliability. It handles error detection and cor- 
rection, and is provided as an appendix to the HL7 specification. 

Implementing HL7 

Most of these concepts were already incorporated into DECrad's ex- 
isting communications functions. Trigger events, a client-server 
model, and most of the basic functions were all in place. Because the 
original DECrad interface package was designed in a modular fash- 
ion, porting the supported functions to the HL7 standard was rela- 
tively easy. All the functions (ADT, Order Entry, etc.) were written 
using batch screens, a software tool that mimics the actions of a user 
data entry screen, handling the application level validation and filing 
of data. Changes mosdy involved writing the Lower Level Protocol 
and applying the HL7 encoding rules, then presenting the data to the 
existing batch screens. 

The query function was a new concept for DECrad. Results were 
typically sent to the HIS as the diagnostic reports were approved, i.e. 
unsolicited updates. This method required the HIS to store large 
amounts of duplicate data, usually for some preset period of time, or 
until the patient was discharged. Using the real-time query for results, 
the HIS or any other system can ask DECrad to send results as an im- 
mediate response to their query. These can be sent in a display for- 
mat, where the receiving system does not interpret the data content, 
but displays the lines of data exactly as they are received. The re- 
sponsibilty of formatting the data is left to the system where the data 
originated, which is best equipped to handle it. Radiology results can 
be displayed in a format that closely resembles a radiology report. 
This function could also be implemented with the RIS as the query- 
ing system, so that RIS users can see results from other systems, such 
as laboratory or pathology. 
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Other Radiology Systems 

In addition to the connections to hospital ADT, orders, and billing 
systems, DECrad has developed interfaces to two other systems 
within radiology. Both of these interfaces use some of the existing 
trigger events developed as part of DECrad' s Events Reporting mod- 
ule. 

The first was an interface to a digitized dictation system. DECrad 
produces management reports that allow the progress of diagnostic 
reports to be tracked and delays in turnaround time to be analyzed. 
But one of the major milestones in the process, the time that the re- 
port was actually dictated, was not captured. As computer based dic- 
tation systems replaced tape recording systems, it was possible for 
the RIS to communicate with those systems, and obtain the missing 
piece of data. One question was how the dictation system would 
identify the examination being dictated. Using the complete exam 
trigger, DECrad sends a unique exam identifier (accession number) 
to the dictation system. The radiologist then inputs that number, in 
some cases using the barcode label produced by DECrad, to associ- 
ate the dictation with a specific exam. The dictation becomes a trig- 
ger event for the dictation system to send a transaction back to DE- 
Crad, containing the dictation date and time. 

More recently, DECrad was interfaced to a radiologist's workstation, 
which uses a speech recognition subsystem, allowing the radiologist 
to create the ASCII text report from spoken words or phrases. This 
interface was developed using the HL7 Version 2.0 specification, 
with extensions to handle free text results (HL7 reserves segment ID 
codes beginning with "Z" for locally defined messages). 

The data flow of the implementation again started with a transaction, 
triggered by completing an exam, from DECrad to the workstation. 
Since the HL7 results message includes patient information, this 
transaction serves as a patient registration, and contains all the neces- 
sary exam information. Once the patient and exam are in the worksta- 
tion database, the report can be dictated, which in turn triggers the 
transaction to send the report text back to DECrad. DECrad processes 
the report text and files it along with other data, such as the radiolo- 
gist's name. If an approval flag is set, then the report goes directly to 



81 



an approved ("final") status in DECrad. The workstation, with rela- 
tively limited database storage, can delete the patient/exam data as 
the report is received by DECrad, or when it receives an approved 
report transaction from DECrad (for those reports that may have 
been dictated outside the workstation). 

Other Standards 

Two other interface standards are of particular importance to radiol- 
ogy. The ACR-NEMA organization, which has developed a standard 
for transferring digital images, has established Working Group VHI, 
whose charter is to develop an application level standard for data ex- 
change between PACS, RIS, and HIS. This group has developed an 
Entity-Relationship database model for radiology and, from this, cre- 
ated some suggested extensions to the ACR-NEMA imaging standard 
data dictionary. They are currently working on a functional model for 
radiology to define the necessary trigger events, and to create a com- 
mand structure that will handle these events. 

The IEEE - MEDIX (Medical Data Interchange) group is working 
on a general healthcare interface standard. In many ways the group's 
focus is the same as the HL7 group. But, MEDIX has adhered strictly 
to the ISO/OSI standards, while progressing in a more deliberate 
manner than HL7. A convergence between the HL7 and MEDIX 
group is expected in the next few years. 

Conclusion 

Much progress has been made in creating interfaces between hospi- 
tal systems. The effectiveness of sharing data through networking 
has proven to solve many of the problems created by independent 
systems. Development of RIS interfaces has historically progressed 
in reaction to specific needs. But, using past experiences, the stan- 
dards committees are making the process more proactive, and are at- 
tempting to create standards that fulfill future needs. 

The creation of application level standards, and the development of 
open systems networks should greatly decrease the effort and cost of 
connecting an RIS to other systems. 
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BERKOM/RADKOM - A Pilot Project in Berlin to improve 
Fast Image Communication between Two or More Hospitals 

F. Astinet, M. Langer, J. Hunger, et al 
Department of Radiology, UKRV-Charlottenburg, 
Free University Berlin, WEST GERMANY 

G. Pack and D. Kuritke 
Siemens AG, Medicine Technics 

B. Scharnberg 

Philips Medical Systems Berlin 

The BERKOM/RADKOM project 

Within the scope of the radiologic communication project 
(RADKOM), sponsored by the German Bundespost, em- 
bedded in the framework of BERKOM (Berlin communica- 
tion project), PAC-Systems were installed in the two diffe- 
rent departments of the Rudolf- Virchow Umversity Clinic at 
Charlottenburg and Wedding (Berlin), somewhat more than 
10 km apart. 

Aims of the project are to prove whether a fast ISDN-B 
test net with 14-0 MBit/s signal speed is suited for a high 
quality transfer of radiologic images and if the network is 
able to establish effective medical teleconferences. 
A further objective is to prove how a digital connection 
between two PAC-systems located in two different radio- 
logical clinics helps to increase as well diagnostic accuracy 
as the effectivity under economical aspects. 

Current status 

In this moment the PACS-Workstations are connected via 
an ETHERNET ( ACR/NEMA-SPI compatible, 10 MBit/s) with 
four CT-scanners and two MRI units. 
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Communication and Image Tranfer between both PAC 
Systems works via an ISDN-B (140 MBit/s) broad band 
network. Direct consulting between two radiologists will be 
possible via monitor coupling (4 high-resolution monitors 
with 1024x1024 matrix) and a videotelephone. 
Up to now because of a lacking high capacity gateway (140 
MBit/s) between PACS-Computer and ISDN-B , still in 
status of development until summer 1991* Datatransfer is 
not fast enough for an istantaneous image transfer during 
the conference. The actual effective transfer rate amounts 
about 170 kBit/s. So the transfer of one image with a 
512x512 matrix takes about 13 seconds. This situation will 
be improved by new software programs with accelerated 
access on the shortterm and longterm archive as well as 
by implementation of fast gateways, which will comunicate 
bidirectionally between both PACS-computers, without 
interconnetion of the LAN (local area network ETHERNET) 
with a bidirectional data transfer rate of circa 2 x 64 
MBit/s. This leads to an expected acceleration of the data 
transfer for a factor of about 400. So a command proces- 
sing, which enables a simultaneous image manipulation and 
demostration, followed "live" by the college at the other 
department should be implemented too. 

Until a solution of the technical problems for the installa- 
tion of a digital command processing will be found it is 
planned to install a temporary arrangement in form of a 
synchroneous cursor operation via video channel on one or 
two of the monitor screens. 

Therefore the conference has to be prepared by back- 
ground image transfer, which does not interfere neither with 
the routine-work on the single dianostic systems (CT and 
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MR) nor with the postprocessing on the PACS reporting 
console. 

The archiving system includes a winchester-local disc 
(1 GBbyte) for shortterm archiving and two digital optical 
discs (DODs) with a capacity of 4 GByte for a longterm ar- 
chiving. 

After 12 months of experience with PACS to PACS commu- 
nication it was proved that an effective data transfer rate 
of only 170 kBit is too slow for a clinical routine work. 
A face to face communication in form of an audio-video 
conference, like it was installed temporarly in our clinic 
during summer 1989 as well as a command processing, al- 
lowing a simutaneous image manipulation, are able to reduce 
misunderstandings between both conferencing partners 
and are absolutely necessary for a successful PACS 
to PACS communication. 

The possibility of simutaneous image reporting from diffe- 
rent modalities contains not only the advantage of compo- 
site analysis of different patients examination with an im- 
proved diagnostic accuracy but in addition a composite 
imaging leads to a composite hardcopy, assembling all the 
relevant images for the clinical physician. This reduces film 
numbers and cost in the radiological department. 
In fall 1990 image postprocessing will be improved by a 
PACS-integrated software program with secondary two- 
and threedimensional reconstructions, regions of interest 
(ROD and time-density curves (TDC). 
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Conclusion 

1. For an effective PACS to PACS communication, integra- 
ted in the routine work of a radiology department, a fast 
broadband network with a data transfer rate around 100 
MBit/s must be available. 

2. Beside the fast image transfer an audio-visual communi- 
cation system is indispensable. 

3. A command processing for simultaneous image display 
and manipulation on all connected systems is desiderable. 

4. PACS should not be limited on archiving and communica- 
tion but offer also the possibility of intelligent image post- 
processing. 

5. A complete archiving system should have a possibility of 
integration of analog data in form of a high resolution digi- 
tizer and in addition a documentation system for patients 
history . 
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INTRODUCTION 

The clinical needs for image capture fall into the principal areas of teleradiology, 
archiving, teaching and research. Captured images may be used for quality 
assurance of imaging devices, for transferring images to a PACS or teleradiology 
system, or for printing images on a laser film printer. The operational need for a 
system for transferring images from scanners to other computers may often center 
around the speed of image transfer, if the capture is to be done simultaneously 
with image photography, or with scanning with rapid reconstruction and display. 
Another critical item is ease of use, which includes a method for ensuring 
reliability of demographic information. Finally, and most important, there must be 
the appropriate level of fidelity of the images, for the window width and window 
level being used. 

In the future we will have digital connections from all our newer digital 
modalities. This will be in the future, however, because of the slow adoption of 
the ACR/NEMA standard. Perhaps a bigger factor is the large number of scanners 
currently installed for which digital interfaces will never be available. Thus, for 
some imaging devices, video image capture will always be the only interface 
method available. 

METHODS AND MATERIALS 

Evaluation procedure 

Five commerically available image capture systems were evaluated: 
Philips/AT&T CommView®System, Adaptive Video, Advanced Video Products, 
Epix, and Images-on-Call™. CommView is a registered trademark of AT&T 
and Images-on-Call is a trademark of Devices and Services Co. 

The evaluations were done using a single CT9800 scanner. A series of digital 
images were programmed on the scanner and used as phantoms for the evaluation. 
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The technique used to make these images available for capture is one which can be 
used with almost all imaging devices. The images are stored in the form of a 
routine patient examination on the archive media which are standard on the 
imaging computers. The method requires only the ability to read and write the 
archive media on any computer which can be programmed. Ideally, the scanner 
manufacturer would make appropriate images available in archive format. 

These images have been designed to be simple in structure, and can be easily 
programmed. In addition, they are available on tape and floppy disk in a variety 
of formats. They are easily understood on the display at the scanner, so the user 
can verify that they have been displayed correctly. After images are captured, 
they are displayed on the standard display on the receiving system, either a 
teleradiology system or a PACS display workstation. 

The scanner was connected to the image capture devices with a thirty foot double- 
shielded 75 ohm coaxial cable. One device came with a five-foot video cable with 
four coaxial connectors, which was used instead. The resulting digital images 
were transferred to a single computer and analyzed there. 

Test images used 

These evaluations were done with a series of digital images, displayed on the 
CT9800, using the standard display program. All of these analyses were done 
using a similar procedure. The images were either programmed on a PC, or on the 
CT scanner itself. Each image was then displayed on the scanner with full 
resolution and accuracy, and the window width and window level controls set 
appropriately. The image was then captured by the image capture device and 
stored digitally. Images were transferred to the another PC and the image 
converted (without error) to another data format which is used for display and 
analysis. 

SMPTE Test Pattern ~ This image was probably originally developed in digital 
format by Joel Gray at the Mayo Clinic, and given to us by Jerry Gaskill, at 
George Washington University, who got it from the Center for Devices and 
Radiological Health. 

256 shades of gray - This image presents all 256 shades of gray with equal areas 
(except for black, which predominates). The shades of gray are arranged in 256 
contiguous areas which are each small rectangles, wider than they are tall. 

Eight Bars — This image contains eight vertical bars that cover the whole image 
vertically. Each bar is 512 pixels tall and 64 pixels wide. The darker bars are at 
the left side of the image and the lighter bars are at the right side of the image, 
being separated in brightness by one display unit each. 
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Uniform field - This image is a flat field with all pixels the same value. In use, 
the window level control is used to set the image brightness to a certain value, 
usually not 128, because of the nonlinearities which may occur at this point. 

Horizontal 256 ramp - This image contains a set of vertical lines, progressively 
brighter as they are closer to the right-hand side of the screen. Each brightness is 
repeated once, so the 256 brightness levels are spread out evenly across the screen. 

Vertical 256 ramp - A ramp which changes in brightness in the vertical direction. 

Alternating BW with stripes -- Alternating black and white vertical lines, each one 
pixel wide. Two vertical stripes, each 10 pixels wide, are used for calibration of 
the full black and the full white values. 

Alternating BBWW with stripes ~ The same alternating pattern except each 
vertical line is two pixels wide. 

Black and white half-screens - These images are all black on one-half of the 
screen (i.e., left or top), and all white on the other half, and may be used to 
calculate MTF. 

Clinical CT scans - Several CT scans were selected to show subtle features. 
RESULTS 

Philips/AT&T CommView 

The most exhaustive testing was done on this system, since it is an integral part of 
the PACS system installed at the University Medical Center. The image capture 
system serves three scanners and in located in a computer room remote from all 
the scanners, and about 400 feet from the CT9800. It was thus tested with video 
cables much longer than the other systems, with a commensurate degradation in 
performance. 

The 256 shade of gray image was captured and its histogram displayed. After a 
spike for the black portions of the image, this histogram should be precisely flat, 
with each shade of gray having exactly 576 pixels in it Instead, the darkest 
portion of the image varies considerably, and the lightest half of the image has an 
oscillation with a period of seven shades of gray. 

The sawtooth appearance of the histogram was caused by the interaction of the 
physical arrangement of the grayscale steps, the limitation in dynamic range, and a 
vertical shading in captured images caused by a hum filter. 
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More recently, we have received an updated version of the frame grabber card, 
which is now a pre-production prototype. It contains a daughter board which has 
faster video circuitry and a more sophisticated method of synchronizing with the 
imaging modality video source. It has redesigned circuitry with a faster video 
amplifier, a DC offset voltage adjustment, and better ability to synchronize with 
the scanner video signal. A new video preamplifier with a wider bandwidth was 
also installed. In addition, a higher quality video cable was installed between our 
CT scanner and the video frame grabber (Belden 8281) and the video distribution 
amplifier was removed and replaced with a much more sophisticated amplifier 
developed by AT&T specifically for this application. It allows the service 
engineer to adjust the characteristics of the amplifier to match it to the particular 
length of coaxial cable installed, compensating for high frequency losses which 
occur in the cable. 

We also installed a fiber optic link with coupling amplifiers which convert the 
video signal on coax to a signal compatible with the fiber optic cable, and convert 
it back again at the receiving end (Merit, Inc.). 

The combination of the new frame grabber circuitry and the wider bandwidth 
video cable/amplifier combination greatly improved the capability of the system. 
The 60 Hertz hum filter was no longer required, so the captured images have very 
little vertical shading. There is still a small amount of noise, but the horizontal 
banding previously seen in the images is not visible unless images are viewed at 
extremely narrow windows. The system now synchronizes correctly with 
alternating black and white lines, with slight shifts in phase being seen as small 
variations in the density of the lines, causing horizontal streaks. When the viewing 
window is narrowed greatly, this variation is easily seen at the upper and lower 
ranges of the gray level value scales. 

Alphanumeric characters displayed over clinical images are now captured 
perfectly at all settings of window width and window level on the CT scanner. 
Although a small amount of blurring at the edges of the characters can be seen at 
extremely narrow window settings, this does not interfere with a computer 
algorithm for the automatic recognition of characters. Therefore, using this 
improved circuitry and cabling, it is now possible to automatically read the name 
of the patient, date of the exam, and other demographic information directly from 
the image at the receiving end of the image capture. This may be used to 
supplement information typed by the operator, or to verify that the information 
typed by the operator is correct, thus preventing incorrect patient identification. 

As expected, the results using the fiber optic link were even better than those over 
coaxial cable. The phase errors seen in the alternating black and white line 
phantom were nearly totally eliminated, so that the noise at the extremes of the CT 
number range was lowered to about 10 grayscale units. 
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It now appears that coaxial cable will be adequate to support all clinical and most 
research uses of images captured by the video frame grab techniques over short to 
intermediate cable lengths, 400 feet at our installation. In the event that longer 
cable lengths are necessary, transmission of the video over fiber optic is a viable 
alternative. 

Adaptive Video 

The Adaptive Video system is a standalone device, not requiring an external 
computer. It has an attached hand-held controller, and is designed as an image 
interface, commonly used to drive another device, such as a laser image printer. It 
synchronized perfectly with the CT scanner pixels at factory settings, with a slight 
limitation in slew rate, probably because the slower of the video input ports was 
being used. Its synchronization was stable across the image. This resulted in 
accurate grayscale rendition and perfectly sharp text. The text was so reliable than 
an automatic character recognition program to read the patient name and ID 
information from the image and store it in a database was easily developed. 

Advanced Video Products 

The Advanced Video Products (AVP) system evaluated is a component of their 
DICAP 1280 teleradiology system. The image capture is performed by a single 
card in a 286-class computer. The system would synchronize with the pixels when 
the cover of the computer was removed and a potentiometer adjusted. This 
adjustment was stable within clinical tolerances, although some drift could be 
detected with the alternating line image. 

EPIX 

While EPIX, Inc. manufacturers many different models of video image capture 
cards, the one evaluated was a SILICON VIDEO MUX with 1 megabyte of image 
memory and a 30 MHz maximum clock rate. It is a single board which fits into 
any computer with a PC/AT bus. It was configured with four coaxial connections 
to the CT scanner, including video, pixel clock, horizontal drive, and vertical 
drive. The S VIP software provided allows the user to configure the device for the 
video format of the CT scanner. 

As expected with the pixel clock connection, the image capture was reliably 
synchronized with the pixels in the CT scanner image. There was a minor 
limitation in the slew rate of the multiplexor on the card, and the device could not 
switch completely from black to white in a single pixel. 
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Images-on-Call 

The Images-on-Call system, from GE Medical Systems, is based on a small stand- 
alone image capture box, approximately 2 x 8 x 10 inches, which can be easily 
located near the scanner video source, with a remote hand-held controller with 
LCD display and alphanumeric keypad. In normal use, it is connected to a modem 
and used for teleradiology, but in this evaluation it was connected to a PC using an 
RS-232 cable so images could be captured to disk locally. 

Of the systems tested, it was unique in its ease of use. It is designed to 
automatically adapt to the video format being used, and so requires no manual 
adjustments to match the particular scanner. In fact, the device could be moved 
from an NTSC video source to a CT9800 without even telling the device that a 
change was being made. It would automatically sense the video characteristics of 
the CT scanner, and adjust its matrix size, synchronization, and acquisition voltage 
levels to match the CT image. 

Despite its small physical size, the performance of the unit was impressive. It had 
a low noise, with a range of 2 grayscale values and yet was fast in switching from 
one video level to another. It never drifted out of calibration, since it adapts to the 
video each time it is used. It uses a matrix size of 500 x 524 for CT9800 video, 
and a grayscale range of 240 so some of the image phantom tests could not be run. 
This limitation causes slight blurring at the edges of some alphanumeric 
characters, but does not interfere with the device's stated use in teleradiology. 

CONCLUSIONS 

The factors determining the quality of captured images predictably include short 
lengths of coaxial cables, high quality cable with excellent shielding and low loss, 
which gives the cable wide bandwidth. The video capture system must not effect 
any other imaging component, such as displays or image hardcopy printers. 
Configurations which did not require the use of video amplifiers tended to give 
better image quality. Loose or noisy cables proved to be a frequent source of 
problems. 

There are many reasons for taking the trouble to align the pixels as captured 
precisely with the pixels as displayed. First, the best grayscale accuracy can be 
obtained only with proper alignment and digitization. It may be possible to 
perform quantitative analysis on properly captured images. The alphanumerics 
will be crisp only if the alignment is correct and reliable across the field of the 
scan. This makes them easily readable and fully professional in appearance. It is 
also possible to apply optical character recognition techniques on the captured 
images for validation of demographic information. 
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Although there are many benefits to using video image capture, there are also 
some drawbacks. Images are captured in 8-bit format, so one must set the window 
width controls to trade resolution for range, and several captures may be required 
per image. The transfer of ASCII demographics is not automatic, and there must 
be some manual intervention after the viewing controls have been set. The 
duration of capture may be annoyingly long, and any misadjustment of the 
equipment may lead to glaring pixel edge effects, such as in the image 
annotation. 

The advantages of video image capture begin with the convenience and 
availability of interfaces for a large number of different scanners, and go on to 
include capturing images which are pre-windowed and ready for display. The 
interface method can be very fast, in terms of the scanner time required to 
accomplish the capture. Also, the capture system may offer multiple outputs, and 
multiple capture systems may be installed, to get around the limitation of a single 
digital output port, such as ACR/NEMA, which could be used to drive a laser film 
printer, or a PACS system, or a teleradiology system, but probably only one at a 
time. 
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Introduction 

Telecommunications systems and networks for image 
transmission have been used over the past four years 
within the Mayo Clinic to support clinical practice, 
research, education, and administration. Some examples 
of these systems include interactive real-time video for 
transmission of imaging examinations recorded on x-ray 
film, transmission of real-time ultrasound procedures, 
transmission of patient examinations in a consultative 
practice of medicine; console-to-console links between 
CT and MR systems; and, compressed digital image 
transmission of a variety of examinations. The networks 
include coaxial cable for local area networks, fiber 
optic links for intermediate distances, and a KU band 
satellite transmission network for long distance 
communication between Mayo Clinic in Rochester, 
Minnesota; Jacksonville, Florida; and Scottsdale, 
Arizona. In addition, the phone lines are used for 
transmission of compressed digital images between and on 
all three campuses. 

The utilization of image transmission in Diagnostic 
Radiology has increased significantly with expanded 
imaging technology . * » 2, 3 This study was performed to 
validate image quality and determine relative accuracy 
of different transmission modalities within the Mayo 
Clinic Telecommunications System. 

Methods and Materials 

Signal-to-noise, resolution, and image quality were 
compared using Ektascan-phone line [Kodak Ektascan Image 
Transmission System; Tl/Rl Software, Version 1.30; 
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Rochester, NY] (compressed digital), Proview-f iber 
optic, and Proview-satellite [General Electric Proview 
Plus; Milwaukee, WI] (real-time interactive video) 
transmission of intravenous urograms. 

Signal-to-noise measurements were made utilizing the 
"spatially-adjacent noise matching 11 technique. The video 
source was connected to a Tektronix Model 1430 Noise 
Measurement Set. The feed-through output of the 1430 
was then connected to a Tektronix Model 1481R waveform 
monitor. Precision, weighted noise generated by the 
1430 was injected into the video signal and matched to 
the noise level of the source device. This was 
typically done on line 21 of the video signal. 

The resolution of the systems was evaluated using 
two different test patterns. The first was a 
conventional monochrome high contrast reflectance 
pattern calibrated in TV lines. The second pattern was 
a radiograph of copper mesh which is conventionally used 
for evaluation of fluoroscopic systems in diagnostic 
imaging. All transmission systems were evaluated using 
monochromatic monitors . 

Intravenous urograms were selected to demonstrate 
image quality and accuracy of detection of various 
grades of tubular ectasia. ^ Collecting duct dilatation 
was chosen for study because of the spectrum of 
involvement from minimal to severe with the thought that 
differentiating minimal involvement from normal would 
test the limits of the different transmission systems. 
Sixty normal and 140 abnormal urograms were graded, 
randomized, and reviewed independently by three 
observers (RRH, BFK, BW). Digital ized images 
transmitted via phone line and real-time video images 
transmitted via fiber optic and satellite systems were 
utilized for the study. Each observer was required to 
indicate grade of tubular ectasia, confidence level, and 
quality of image. 

Grading of the cases was determined by degree of 
dilatation of collecting ducts with Grade 0 being 
normal, Grade 1 mild dilatation, Grade 2 moderate, Grade 
3 marked, and Grade 4 severe. A total of 200 
representative films were selected and graded. The 
cases included 60 normals, 60 Grade 1, 56 Grade 2, 16 
Grade 3, and 8 Grade 4. The cases were then randomized 
to one of four groups labeled W, X, Y, or Z. Each 
group contained 50 cases composed of 15 normals, 15 
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Grade 1, 14 Grade 2, 4 Grade 3, and 2 Grade 4. 
Reviewers were unaware of the relative number of each 
grade number in each group. 

The experiment was designed so that three 
uroradiologists each reviewed two groups of film on 
every modality for a total of 100 cases. One group of 
films was seen on every modality by every reviewer 
(Group W) while one group of films was seen by each 
observer only once (Group X, Y, Z). 



METHOD OF TRANSMISSION 



Ektascan Proview Proview 

Re v iewe r Ph one Line Satellite Fiber Op tic 

1 XW YW ZW 

2 ZW XW YW 

3 YW ZW XW 



This arrangement allowed us to evaluate consistency on 
each reviewers part, consistency among the reviewers, 
and variability on different transmission systems . For 
each case the reviewer evaluated Grade 0 to 4, 
confidence level as uncertain or certain, and quality of 
image as poor, adequate, very good, or excellent . The 
reviewer could optimize the image by whatever options 
were available on each piece of equipment. No time 
limit was imposed. The pre-graded standard cases were 
available for review during evaluation. 



Discussion 



A signal-to-noise ratio of 60 db was considered an 
excellent image with the noise undetectable to the human 
eye. The Proview system was determined to have a 
signal-to-noise ratio of 47 db. The fiber optic link 
did not add significant noise. Non-encoded satellite 
transmission was excellent at 58 db. With the necessity 
of encoding or scrambling the image, additional noise 
was generated decreasing the signal-to-noise ratio to 48 
db. The signal-to-noise ratio of the Ektascan system 
was not considered reliable because of the low 
resolution which produced blurring. The blurring of an 
image results in blending each image point with its 
surrounding area. The effect was to smooth out noise 
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and make it less visible. 

The resolution of the Proview imaging system for the 
high contrast (reflectance) pattern was determined to be 
500 TV lines. Some loss of resolution was noted when 
the video signal was transmitted over the satellite 
decreasing the resolution to 450 TV lines. The 
resolution of the Ektascan system on the high contrast 
pattern was 250 TV lines. However, the Ektascan 
resolution using the low contrast pattern was 
approximately equal to the Proview system at 24-30 lines 
per inch. This result might be anticipated since it is 
necessary to filter the analogue video signal prior to 
digitization. The filtration will primarily affect the 
high frequency components. The low contrast resolution 
pattern will be affected most by noise. In fact, 
filtering the analogue video signal prior to 
digitization will decrease the noise in the signal 
providing better low contrast resolution. 

Measurement of signal-to-noise and resolution 
provided an objective evaluation of the different 
systems. Whether there was any subjective difference in 
the systems was the next question to be addressed. The 
grade assigned by each reviewer was compared to the 
pre-experiment assigned grade and tabulated for each 
reviewer on each method of transmission. A trend in the 
number of correct responses was noted when comparing the 
systems. The Proview-f iber optic system had the most 
correct responses. There was, however, no statistically 
significant difference in detection of normal versus 
abnormal or in determination of the severity of 
dilatation within one grade. Despite disagreement or 
variability in grading amongst the observers, in 98% of 
the cases they agreed within one grade of the 
pre-assigned grade and in 87% of the cases the correct 
diagnosis of abnormal versus normal was made. 
Individual performance on each method of transmission 
was also considered. While two reviewers were more 
accurate on the Proview systems, one reviewer had the 
opposite results. Combining the results, no 
statistically significant differences were identified. 

The kappa statistic for each method of transmission 
was calculated. The kappa statistic is similar to the 
correlation coefficient and measures the ability of the 
reviewers to agree with each other. Values less than .4 
show no real agreement, values between .4 and .75 show 
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good agreement, and values greater than .75 show 
excellent agreement. The kappa values obtained range 
from .44 to .55. 



Ektas can-phone line 
Proview-satellite 
Proview-f iber optic 



KAPPA VALUES 
W 

0.44 (0.33-0.55) 
0.46 (0.36-0.57) 
0.54 (0.43-0.64) 



0.45 (0.35-0.56) 
0.55 (0.45-0.65) 
0.50 (0.39-0.60) 



These values are at the low end of the range and overlap 
each other. These values indicate that there was more 
variability between reviewers than there was between 
methods of transmission. 

Reviewers also evaluated quality of images and rated 
them as poor, adequate, very good, or excellent. The 
Ektascan produced more images rated poor and fewer were 
perceived as excellent than did the Proview systems. 
There was virtually no difference between the 
Proview-f iber optic and Proview-satellite system. There 
was no correlation between the perceived quality of the 
images and the percent correct for any of the 
modalities. Similar results were obtained for all 
groups of film evaluated. The confidence of the 
reviewers varied directly with the perceived quality of 
image for all methods of transmission. There was, 
however, no significant difference in accuracy. 

Conclusion 

There was no significant difference in accuracy of 
interpretation using transmission of data via 
Ektascan-phone line, Proview-f iber optic or 
Proview-satellite. There was more variability between 
reviewers than between methods of transmission. The 
reviewers were more confident of their assignment of 
grade when the quality was perceived as high. Although 
there were general trends showing the video images to be 
better than compressed digital images, the results did 
not reach statistical significance. The reviewers all 
expressed a subjective preference for the images 
generated on Proview and transmitted in a real-time 
interactive video mode. 
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Abstract 

A powerful, low-cost, 386 based image workstation 
Tele-Radiography system TR200 with multi- 
functional capabilities has been developed by ADA 
Digital Systems, Inc. for high performance 
medical image applications. This workstation 
provides an integrated image manipulating 
environment that offers a unique combination of 
capabilities to capture, process, store, 
retrieve, print, and distribute medical images 
through telephone lines. Featuring a versatile 
packaging design, the TR200 can be configured as 
a dual medical image viewing, an image 
communication, an image database management, and 
an image desktop publishing workstation. The 
TR200 workstation provides a video camera and a 
light box to capture images from X-ray films. A 
powerful image processing software package 
provides the tools for image enhancement, image 
measurement, edge detection, image geometric 
transformation, etc. The dual monitor display 
makes it convenient for doctors to compare two X- 
ray images. Using ADA image communication 
software package, images can be transmitted from 
one hospital to another via modems and telephone 
lines. The software makes it possible for 
doctors to discuss the cases of patients when one 
workstation displays the X-ray image of a patient 
while another workstation at a remote site 
displays the same image by getting it through 
telephone lines. The doctors from these two 
hospital can work simultaneously to deal with the 
same image. They can enhance the image, draw 
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polygons and write text on the image using a 
remote image processing technique, i.e. one 
station can be the master which controls the 
slave - the remote station. All the operations 
of the image processing will be simultaneously 
performed on the two workstations. The 
sophisticated image compression, encoding, and 
decoding algorithms of ADA ensure the high speed 
image data transmission and receiving. The 
relational image database of ADA can store and 
retrieve not only the text information, but also 
the image information of patients. Images can be 
printed out on regular papers by a high 
resolution laser beam printer to get clear hard 
copies. The modular design and user friendly 
screen menu interface of TR200 affords the system 
flexible and easy for use. Simplified operations 
are also provided for daily operation. In an one 
key operation, image capture and enhancement are 
performed automatically. Bulk images can be 
selected and sent out at night by a sophisticated 
image communication package without the people on 
duty. Two ADA TR200 Tele -Radiography 

workstations have been successfully installed in 
the Johns Hopkins Hospital at Baltimore, MD and 
the Chamber sburg Hospital at Chamber sburg, PA 
respectively . 



INTRODUCTION 

In medical imaging applications, an integrated 
system which facilitates those functions as 
capturing , processing, communication, etc. will 
be widely used in the field of radiation therapy. 
Doctors can capture pictures from films into 
computer. The digital images are enhanced by 
various algorithms to highlight certain 
interested organs or analysis some features. 
Measurements and statistical results are easily 
obtained. It is often the case that doctors in 
local hospitals want to link to medical research 
center so they can benefit from the extensive 
knowledge of academic institutes. Through 
telephone lines, digital images can be 
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transmitted from station to station. Doctors can 
discuss patient's cases on line where displaying 
and processing the same images on computer 
monitor screens. 

ADA Digital Systems, Inc. thus developed a 
powerful, low cost, 386 based medical imaging 
workstation Tele-Radiography 200. The system 
includes a 386 Personal Computer, a light box, a 
camera, an image digitizer and processor, a 
system monitor and an image monitor, a mouse, and 
a modem. First, the film on the light box is 
digitized into computer. It is automatically 
enhanced based on its histogram. A pair of 
images is grouped together displaying on the two 
screens for comparison. Normally one comes from 
simulator, the other is portal film. Bulks of 
images can be prepared and sent to another 
station through telephone lines at the night. So 
doctors in both stations can preview those images 
before discussion on line. During the 

conference, all the operations are in master- 
slave mode, that is to say, the master station 
totally controls the slave. Of cause, master and 
slave can be easily exchanged. 

In the following sections, we will give a brief 
description of the system TR200. 



SYSTEM DESIGN 

For an application system, it should be very easy 
to use and flexible enough to add in more 
functions according to user's needs. 
Although it is easy to add in functions, the 
command driven operation is not good for doctors 
to use it. Doctors almost always found they are 
confused by the strange command names and lots of 
arguments. So we designed a menu driven, modular 
structured interface. 

In the menu, a function is selected by using 
arrow keys to go to the function item. Then it 
is evoked by hit the Enter key. We group those 
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items by their usage. Due to the modular nature, 
it is fairly easy to have a new function added 
into it. Actually, we have more than 200 
separate imaging modules, they can be tailored to 
most specific needs. 

In the development of Tele-Radiography System, we 
cooperative with doctors in hospitals and try to 
do things according the way doctors used to. In 
medical imaging, we find that the cooperation 
between doctors and computer engineers is very 
important. Several functions in the system are 
written to meet the specific needs of doctors. 



FUNCTIONS DESCRIPTION 

1. Image Capturing 

The system grabs one frame in real time - within 
1/30 second. Live image is displayed on the 
screen while it is digitized into the frame 
store. The size of the digital image is 512 by 
480 pixels, one pixel has 8 bits or 256 
grayscales. To reduce the noise, several images 
are averaged in the frame store. The digitized 
images can be stored in hard disk for later 
retrieval . 

2. Image Processing and Editing 

By using image processor, the quality of X-ray 
images can be greatly improved. The technique we 
use is called interactive histogram 
transformation . 

Those traditional techniques such as edge 
detection, high pass or low pass filtering, 
sharpening, etc. are also provided. When two 
images are displayed on the screens, texts and 
polylines can be drawn on the images. 
Measurements of length or area on interesting 
object can be also got while doing calibration 
before measure. 
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A detailed list of image processing and editing 
functions is shown below: 

Image Display 

Scroll, Pan and Zoom 

Rotation or Reversing 

Show Pixel Value 

Drawing Polyline 

Writing Text 

Calibration and Measurement 

Histogram Enhancement 

Image Sharpening 

Edge Detection 

Mean Filtering 

Negative Picture 

Vertical or Horizontal Profile 

Addition or Substraction 

etc. 

3. Image Transmitting 

It is very convenient to prepare all the films 
before conference and send them at the night. 
When images are captured and stored in hard disk, 
doctors can work out which pairs will be showing 
and select the images want to be sent. After 
that, ask doctor at the other end turn on the 
machine and enter the receiving mode. If 
everything is OK, set the time to transmit and 
leave it there. When the time comes, the sender 
automatically dial the phone. If connected, send 
all the images selected. Then hang up. the phone 
and give the message about the transmission. 

During transmission, if there is anything wrong, 
the system will send that part again. Of cause, 
it only tries several times in case of sever 
errors occurring on the line. This will prevent 
from unnecessary connection for a long time. 

Even though we use a 9600 baud modem for 
transmission, it will still take about 5 minutes 
to send one whole picture. However we developed 
a compression utility to reduce the size of the 
transfer data. For most X-ray films, the 
compression ratio is around 1/3. That is to say, 
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the transmitting time is just about 11/2 minutes 
for one image. 

4. Remote Imaging Control 

In terms of remote imaging control, we mean to 
transmit, manipulate, and edit images on the 
screens interactively where two stations is 
linked together by ordinary telephone line and 
they are doing exactly the same thing. So 
doctors in deferent hospitals can sit in front of 
their own computer, look at the screens, control 
the process, and discuss about patient's cases. 

In the system, communication is implemented in 
master-slave mode. Master has the total control 
of the slave. Master can draw polylines, write 
text, or do image processing where the slave 
behaves the same as the master. For example, 
after master has the polylines drawn on the 
screen, the slave has the same polylines appeared 
on its screen. 

For efficiently and correctly transmitting data 
and command over ordinary phone line, we 
developed a packet communication protocol. 
A packet contains all the information about 
transmitting sequence, command, data, and check 
value. When it is received by the other end, 
after checking the received bits, the command is 
executed at the same time with the sender. 

5. Image Management and Printing 

Sometimes, it is useful to store the images and 
related patient data in a database, we provide 
the option to do the image management and 
organization functions. Also the images 
original or processed - can print out by laser 
printer. 



CONCLUSIONS 

Now, the ADA Tele-Radiography System TR200 is put 
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in use at the Johns Hopkins Hospital at Baltimore 
and Chambersburg Hospital in Pennsylvania. 
Because it integrates imaging, communication , 
printing, etc. into one complete system, it is an 
ideal computer assistant for radiology. Doctors 
found it very easy to use and powerful. The 
cooperative development process with doctors is 
a good way to implement such kind of specific 
application system. Now we are work on the new 
function to capture images directly from X-ray 
beam. So we do not need to always take pictures 
on films. 



Camera 



Image 
Monitor 



Personal 
Computer 



Modem 




Analog 























a 





Digital 



Phone 
Line 



Sender 



Camera 



Image 
Monitor 



Personal 
Computer 



Modem 



Analog 



.{•••mm ♦♦♦l^ 



Analog 



f 


















J 






«3> 



Line 



Receiver 



Teleradiology for Remote Evaluation of CT Scans 
at a Rural Hospital 

Jeffrey L Chenoweth and Carroll R. Markivee 
Department of Radiology, Saint Louis University School of Medicine 
and John Cochran Veterans Affairs Medical Center, St. Louis, MO 



INTRODUCTION 

Patients who live in remote or rural 
locations have always had difficulty in 
obtaining access to modern high quality 
medical care. Many are fortunate to have even 
basic services available. Expensive high 
technology imaging procedures such as computed 
tomography (CT) and magnetic resonance imaging 
(MRI) have revolutionized the practice of 
medicine yet remain unavailable due to 
inherent high costs and low volumes in such 
rural areas. In an effort to meet this need 
mobile equipment on trailers towed by trucks 
has been developed which would make the unit 
available on a periodic basis. Unfortunately 
due to distances involved and low volumes of 
exams it often is not possible to have a 
trained radiologist on site for immediate 
evaluation of the images obtained. Even when 
a radiologist is available, he or she may may 
not be trained in the newer imaging modalities 
and would benefit from consultation with 
subspecialists in these areas. 

Teleradiology systems have been proposed 
to help solve this problem. Teleradiology is 
the evaluation of radiologic images 
transmitted from a distant site. We report on 
a pilot program to evaluate the feasibility of 
providing remote interpretation of CT scans 
using a commercially available microcomputer- 
based image transmission system (ITS) . 
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METHODS 

Over a four month period 49 CT scans were 
performed on 41 patients at a rural Veterans 
Affairs hospital at Marion, Illinois and 
transmitted for evaluation at a tertiary care 
veterans Affairs Medical Center in St. Louis, 
Missouri, a distance of 120 miles. 

The CT scans were performed on a CT9000 
(General Electric, Milwaukee, WI) mounted on a 
truck which was available for one or two 
afternoons a week. The examinations performed 
included 19 head, 16 abdomen, 9 pelvis, and 5 
chest scans. The studies were imaged as usual 
with a 12 on 1 format onto a 14x17 in. film 
and subsequently digitized. 

The ITS utilized was the commercially 
available Ektascan (Kodak, Rochester, NY) . 
The filmed CT images are placed on a lightbox 
above which is mounted a vidicon camera with a 
macro zoom lens. The camera is interfaced to 
an IBM PC AT microcomputer with a proprietary 
frame -grabber board. Up to 85 images can be 
stored on the 20 MB hard disk. The images are 
aquired as a 512x512 matrix, but displayed as 
a 512x480 matrix on the separate 
control /display monitor, with 256 gray levels. 
The internal 9600 baud modem transmitted the 
data over ordinary telephone lines to the 
similar receiving unit. The system utilizes a 
2:1 data compression method to allow image 
reconstruction without data loss. The images 
were stored on the hard disk of the receive 
module for evaluation by a staff radiologist 
who would phone the results to the referring 
hospital. Several days later the filmed 
archival images were received for final 
diagnosis. 

RESULTS 

The ITS is capable of displaying an image 
of a standard 14x17 in. film on a 512x480 
matrix. Preliminary experiments showed that 
such images were not adequate for evaluation 
of the typical 12 on l CT format due to loss 
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of detail. The vidicon camera is equiped with 
a macro zoom lens capable of optical 
magnification of selected portions of the 
filmed images. This results in an effective 
increase in matrix size and improved image 
quality (1) . This capability was used to 
transmit each CT image separately. 

A total of 49 examinations were 
evaluated, including both normal and abnormal 
studies (Table 1) . The head CT scans included 
cases of tumor, infarct, and atrophy. The 
chest scans included lung tumors and aortic 
aneurysm. The abdomen and pelvis scans 
included cases of simple and complex renal 
cysts, liver cysts, ascites, aortic aneurysms, 
and tumors of the kidney, liver, prostate, and 
bladder . 



Table 1. CT Findings in 41 Patients 



head chest abdomen pelvis 

normal CT 5 0 2 5 

abnormal CT 14 5 14 4 



infarct 4 

atrophy 8 

mass / tumor 2 4 4 4 

cyst 8 

ascites 2 

aneurysm 1 3 

other 1 5 



Note: several abnormal scans had more than one pathology. 



In this series there were no major 
disagreements between interpretations from the 
transmitted and hard copy images; the site of 
pathology was correctly determined in all. As 
anticipated, the confidence level of the 
radiologists was considerably higher with the 
film images than with the digitized images. 
In particular, distinction of fine density 
differences (such as a simple cyst verus 
abscess) or subtle changes in edge 
characteristics (such as simple versus complex 
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cyst) were often difficult to evaluate on the 
monitor. Transmitting images with region of 
interest density measurements made on the 
original scans helped to some degree in this 
regard. The transmitted images nonetheless 
required more careful evaluation than the film 
images and often needed manipulation with the 
display controls. 

The time element became an important 
factor in the system. Transmission time for 
one image is approximately one minute at 9600 
baud. A standard 22 image head CT would thus 
take about 22 minutes to send, a standard 
chest CT could take 50 minutes or more. The 
software can be set for unattended 
transmission and reception. 

Another time factor of critical 
importance is the time required for the 
radiologist to evaluate the transmitted 
images. Only one image is available at a 
time. Although any image can be recalled from 
the hard disk in a few seconds, it can be 
tedious to review a whole set of images, 
particularly as many required further 
manipulation with the window/level, zoom, or 
histogram equalization controls. Also, 
radiologists are used to quickly scanning a 
film to construct a mental 3D spatial 
relationship. This is considerably more 
difficult when all the images are not visible 
at once, in addition, more care and time is 
needed to compensate for the relatively 
decreased detail available on the ITS. 

DISCUSSION 

Teleradiology systems have been available . 
for over three decades (2) . They have been 
used for transmission of standard radiographs 
obtained in hospitals, emergency rooms, and 
clinics (3-6) . A natural extent ion of this 
developmental process is the remote 
transmision of Examinations such as CT and MRI 
scans. Many of the previously reported ITS 
are impractical due to expensive or one of a 
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kind equipment configurations. The 
development of microcomputer-based ITS have 
now resulted in commercially available units 
at a reasonable cost. The simultaneous 
development of mobile CT and MRI scanners has 
made it possible to offer these services to 
patients in remote or rural locations. 

This pilot program was designed to 
evaluate whether such a service was feasible 
from a diagnostic and economic standpoint. We 
were encouraged by the agreement in 
interpretation of the transmitted and filmed 
images. Nonetheless, further work is required 
to evaluate the sensitivity and specificity of 
the ITS in a wider range of pathology, and in 
particular in the evaluation of subtle 
alterations in tissue density. Such 
differences are essential to the radiologist's 
confidence in image evaluation. Since the 
digitized images are derived from an analog 
filmed image, there is of necessity some loss 
of diagnostic information, as has been shown 
in transmission experiments involving plain 
radiographs (4,5,7-9) . Direct transmission of 
the original CT data might overcome this 
limitation, but would require dedicated 
specialized interface equipment. 

Several methods of data transmission have 
been employed in the past, including direct 
coaxial cable, microwave, and satellite 
communication (2,3,10). Telephonic links as 
used in our study have the advantage of low 
cost and nearly universal availability. The 
prime drawback to this method is the time 
required to transmit the large volume of data 
required for CT scans up to an hour at 9600 
baud for a standard chest CT. Data 
compression algorithms can cut the time 
requirements, but at the cost of image 
quality. In the future, integrated services 
digital network (ISDN) , a developing 
telecommunications data standard, offers the 
promise of much greater data rates while 
preserving compatability with current 
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telephone wiring (11) . 

The second goal of our project was to 
evaluate potential for improved patient care 
via this system. Previously patients had to 
be transported 120 miles to a tertiary care 
center to obtain a CT scan. Such travel was 
difficult for the sick inpatient. It was also 
a burden on the outpatient who had to miss 
work. Both groups had to be hospitalized 
overnight before the return trip, further 
increasing costs. The immediate evaluation of 
the images allowed more timely patient care. 
The number of abnormal exams in our series 
demonstrates the need for such a service. 

We have been encouraged by these 
preliminary results and plan to continue 
further evaluation, particularily in regard to 
sensitivity and specificity of subtle lesions. 
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Introduction 

Frame grabbers are frequently used in medical imaging 
systems to convert analog video signals into a digital matrix for 
viewing. The applications using this technology include tele- 
radiology, ultrasound, nuclear medicine, and teaching systems. 
While the quality of images captured by frame grabbers is gen- 
erally less than those generated by laser scanners, the sys- 
tems are considerably less expensive and are believed to be 
have adequate resolution and gray scale range for many appli- 
cations. 

As part of a research project to digitize cardiac angiograph- 
ic images from 35 mm cine film, we needed to develop an inex- 
pensive system to acquire and store these images for computer 
analysis. The Macintosh II computer (Apple Computer) was 
chosen as the developmental platform. Since there were a wide 
variety of high-resolution, commercial, 8-bit frame-grabber 
boards available for the Macintosh system ( as well as for other 
small computers) with similar specifications (640 x 480 frames 
from RS170 video signals) and little information in the literature 
on their suitability to medical imaging systems [1-3], we have 
made a comparison of four state-of-the-art boards suitable for 
our application. All board were capable of digitizing and dis- 
playing an entire image in real time (1/30 sec) for processing on 
a personal computer. 
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Materials And Methods 

Frame grabber boards and their accompanying software 
packages were obtained from the following manufactures: 

A) Quick Capture Board (DT2255-60Hz), Data Translation 
(Marlboro, MA). 

B) Image Grabber Board, Neotech Group Limited, 
Camberley, Surrey, England. 

C) NuVista Board with 1 MB internal memory and 
Truevision NTSC VIDI/O Box (encoder/decoder), Truevision, 
Inc., Indianapolis, Indiana. 

D) Video Image 1000, Scion Image Systems, Walkersville, 
Maryland. 

The Scion board did not contain an internal buffer mem- 
ory; it used the host computer's NuBus to store images directly 
in the computer's memory. This board was unique in that it 
could display true live time video images on the Macintosh 
screen (with as small truncation in image size). Boards A, B, 
and C captured full size images: boards A, B displayed images 
close to real time on any appropriate monitor connected to the 
computer; board C connected directly to the Apple 13 inch mon- 
itor and was not able to display live images during capture. 
Each board occupied one expansion slot on a Macintosh II 
computer (8MB memory, E Machine TX color monitor and Apple 
13 inch color monitor, 8 bit video cards). Input video signals for 
the boards were generated from 3 sources: 1) PDP-11 Nuclear 
Medicine Imaging System (PCS-512, Picker International), 2) 
TSG-51 Video Pattern Generator (General Electric) with a 
checker board test pattern with gray scale ramps, and 3) a cus- 
tom cine film (35 mm) display system with a Panasonic TV cam- 
era (WV 1500). For this device, a 35 mm gray scale resolution 
pattern (EAI test pattern) was photographed. A cardiac angio- 
graphic image was independently evaluated by five radiologists 
and a test pattern by the four radiologists and one physicist. All 
four frame grabber images were displayed simultaneously on 
the E Machine monitor. Board evaluation was based on image 
sharpness, range of gray scale, and absence of noise and arti- 
facts. 
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Results 

Variations in the ability of boards A,B,C and D to capture a 
full range of 256 gray shades is shown in Fig. 1. Each board 
was connected to the video pattern generator and the resulting 
image was stored. The gray scale ramps are grouped together 
for comparison. 




Figure 1 . Gray scale ramps generated by boards A - D. 



Significant differences between the number of steps and the 
sharpness of the transitions are shown. While board D displays 
the widest range of steps, it does not use the full range of 256 
values and is noisy. This is illustrated by the profiles (Fig. 2) of 
the gray scale ramps. 

Noise was measured in a rectangular region-of-interest 
centered on the middle gray step (Table 1 , A) and in a dark por- 
tion {Table 1 ,B) of each image. The results are shown in Table 
1. 

Table 1 

A B 





mean 


s.d. 


mean 


?.d. 


A 


134.91 


0.32 


211.97 


0.88 


B 


146.10 


0.30 


221.42 


0.53 


C 


156.68 


2.57 


233.04 


1.53 


D 


124.22 


1.15 


192.39 


1.37 



Profile Values of Gray Scale Pattern 
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Pixel Number 

Figure 2. Profiles through gray scale ramps shown in Fig. 1 . 
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A comparison of the ability to resolve lines in the EIA pattern 
is shown in Fig. 3. Once again there is marked variation among 
the boards. 

A B C D 




Figure 3. Resolution pattern from boards A - D 




Figure 4. Ventricular performance image (board A). 
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To test the boards' performance with a nuclear medicine 
image, a ventricular performance image was generated on the 
PCS Computer and grabbed by each board (Fig. 4). 

Details of the image are shown in Figures 5 and 6. Note in 
particular differences in noise patterns and resolution. 
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Figure 5. Comparison of boards A and B 




Figure 6. Comparison of 
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Since the images captured by the boards were going to be 
used for the analysis of various clinical protocols, a frame from 
a coronary arteriogram was grabbed (Fig. 7). 




Figure 7. Coronary arteriogram grabbed by board A. 



The four copies (captured by boards A - D) of the angio- 
graphic image were displayed simultaneously on the E 
Machine monitor and five radiologists were asked to rank their 
image preferences from best (1) to worst (4). Table 2 shows the 
preferences of the observers. 

TABLE 2 



Board 



Rank 


A_ 


_B 


c 


D 


1 


2 


1 




2 


2 


2 


1 




2 


3 


1 


2 


1 


1 


4 




1 


4 
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In additon, the video test pattern was displayed in a similar 
fashion and preferences ranked: 



TABLE 3 



Board 



Rank 


A_ 


-B 


a 


c 


1 


4 






1 


2 


1 


2 




2 


3 


0 


2 


1 


1 


4 


0 


1 


4 


1 



Discussion 

The test patterns revealed large variations in image quality. 
Problem included excessive background noise, vertical noise 
patterns, interlacing and synchronization problems. Overall, 
board A appeared to generate the highest quality images. 
However, the observer tests showed that viewer preference was 
an important factor in choosing a board for clinical applications. 
In general, we concluded that while significant variations exist- 
ed among the boards, these difference were harder to observe 
in standard clinical images. For quantification of regions-of-in- 
terest and the detection of edges, the best quality images are 
required from frame grabber boards. 
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The term "teleradioiogy" refers to the 
transmission of radiographic images from one 
location to another for interpretation, review 
or screening. Most of the work to date # and 
the most demanding, has involved the scanning 
of conventional radiographs to convert them to 
a digital format for transmission and primary 
interpretation by a qualified radiologist. The 
review function usually involves the 
transmission of images with a report for 
viewing by attending physicians in settings 
such as the Intensive Care Units in major 
medical centers. The use of teleradioiogy 
systems for screening typically involves the 
transmission and viewing of images by the "on- 
call" radiologist in consultation with an 
emergency physician to determine appropriate 
care in urgent cases. 

At The Johns Hopkins Medical Institutions 
teleradioiogy systems are being evaluated to 
determine their utility within the Johns 
Hopkins Health System which includes many 
medical facilities at different locations in 
the State of Maryland. When the facilities are 
linked by high speed transmission media, 
medical images from all types of modalities may 
be transmitted for many purposes, particularly 
when a site does not have a radiologist 
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available and expert consultation is needed. 
The current pilot study of a high resolution 
teleradiology system is designed to evaluate 
each of the components as well as the total 
system in terms of technical feasibility, 
clinical acceptability and cost effectiveness. 

The specific clinical functions of concern 
include the provision of diagnostic services to 
an emergency room in an affiliated hospital 
that does not have a radiologist available 24 
hours a day. The study will also include the 
sharing of an optical archive specifically 
designed for digital images from all types of 
modalities. Digitized film, magnetic resonance 
images, computed tomography slices, and nuclear 
medicine scans will be transmitted for storage 
in the "jukebox 11 , from a film digitizer and 
from direct digital connections to selected 
imaging equipment, and retained for subsequent 
retrieval and display. The pilot study is 
expected to provide data that will be useful in 
evaluating the specific equipment comprising 
the teleradiology system and to gain experience 
that is pertinent to the design of a 
comprehensive image management network for the 
Johns Hopkins Health System. 

The major components of the teleradiology 
system include a film digitizer and a medical 
imaging gateway at the Francis Scott Key 
Hospital that is linked by a terrestrial Tl 
line to The Johns Hopkins Hospital where a 
medical imaging gateway, high resolution 
workstation and an image archive are located. 
The film digitizer is manufactured by DuPont in 
Wilmington, Delaware and is designed to provide 
a choice of spatial resolutions at 105 or 210 
micron pixels with 12 bits of contrast. This 
option required the development of a "VDRSC" 
scanner control board and new software 
provided by Vortech in Dallas Texas. The 
medical imaging gateway supplied by Vortech 
includes local storage, buffer memory and 
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communication control functions. Vortech 's 
interface to the DuPont scanner is a DR11W 
connection . 

At Francis Scott Key Hospital there is a 
multiplexer supplied by Potomac Digital to 
control the flow of data between the medical 
imaging gateway and the Tl line supplied by the 
C&P Telephone company* At The Johns Hopkins 
Hospital there is also a Potomac Digital 
multiplexer and a Vortech medical imaging 
gateway with multiple ports. This controls the 
transmission of digital images to a Virtual 
Imaging workstation with two high resolution 
monitors and to an archive containing both 
magnetic and optical storage that utilizes a 
Cygnet jukebox and software developed by 
Vortech. The software 

that facilitates the user functions for the 
Virtual Imaging workstation was developed by 
the Radiology Department at The Hospital of The 
University of Pennsylvania. 

As of this writing, (April 1990) all of 
the equipment is expected to be in place and 
operational by May 14th , at which time a 
controlled study of the diagnostic accuracy of 
the teleradiology system will begin. Films 
demonstrating selected abnormalities and 
matching normal radiographs will be digitized 
at Francis Scott Key and transmitted to Johns 
Hopkins for interpretation on the electronic 
display workstation and for storage on the 
archive. For each case a comparison of the 
interpretation derived from the display will be 
made with the interpretation from the film and 
the data subjected to an ROC analysis. The 
study design also includes a comparison of 
inexperienced and experienced readers in both 
the display and film modes of interpretation. 

Following the controlled study, an open 
program will be instituted in which 
consultation regarding any type of medical 
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imaging examination may be requested by 
physicians at Francis Scott Key that will be 
provided by members of the radiology faculty at 
The Johns Hopkins Hospital. The faculty who 
will interpret the images on the high 
resolution display terminal and provide a 
prompt report. In each case, the original films 
will be read independently and a comparison 
made of the resultant report with that derived 
from viewing the images on the electronic 
display. Any discrepancies noted between the 
two reports will be submitted to a Consensus 
Panel for resolution and the outcome recorded 
in the pilot study database. Through the 
controlled and open studies, it is expected 
that objective data bearing on clinical 
acceptability will be obtained which in 
conjunction with a subjective evaluation of the 
utility of the system by the participating 
staff will provide definitive results. 
Preliminary findings will be presented at the 
Session on Teleradiology at the 10th Conference 
on Computer Applications in Radiology. 
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Requirements for Reference to Previous Images & Reports 
When Interpreting Chest Examinations 

Steven M. Montner, Heber MacMahon, Paul Davis, et al 
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Hospitals and Clinics, Chicago, IL 



Availability of previous radiological examinations is a 
valuable asset in interpreting new examinations. Therefore, retrieval 
of both images and reports of previous examinations is an essential 
requirement for film file systems or for PACS, and such retrieval 
functions must be supported by radiology information systems. To 
gain a better understanding of these requirements, we reviewed the 
standard clinical practice in the chest radiology section of a teaching 
hospital with regard to the use of films and reports of previous 
examinations. 

In our practice we found that previous examinations had 
been performed within 5 years for 48% of the routine PA and lateral 
chest examinations, and for 90% of portable chest examinations. 
Although previous images generally are reviewed, it can be helpful 
to review the reports, particularly when the earlier examinations 
were interpreted by a different radiologist. Even without PACS, a 
radiology information system can provide convenient retrieval of 
previous reports. In our radiology information system, developed at 
the University of Chicago, all reports for the prior five years are 
online. The chest reporting module offers a default suggesting that 
the radiologist review the most recent prior report. Using this 
system chest radiologists chose to review reports in 36% of the 
cases in which they were available. An option to display additional 
prior reports in reverse chronological order allows data to be 
collected by the system to show the distribution of references to 
immediate prior reports, second prior reports, etc. Summaries are 
compiled for individual radiologists to show inter-radiologist 
variation in reporting style. 
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Delays in reporting commonly occur due to the need for 
retrieval of films of prior examinations; we suggest that the 
incremental cost of providing convenient access to previous reports 
may be much less than the cost of similar access to images. In 
summary, accessibility of previous reports, independent of 
availability of the images, is considered important by radiologists 
and appears to support improved accuracy of interpretation. 
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Introduction 

Medical records are likely to be more useful if they contain 
appropriate graphic material. In the case of the radiology report, 
diagnostic conclusions and recommendations become more 
convincing when the clinician personally can review the image 
evidence ("findings") on which these are based. While the x-ray 
film is the usual source of this evidence, it may not be convenient for 
the clinician to match it with the report for purposes of confirming 
the diagnosis, advising the patient concerning his/her health status, 
performing chart reviews, etc. 

Hypothesis 

Modern desk-top publishing techniques make it possible to 
incorporate radiographic images, appropriately selected and 
annotated, as part of the radiology report. We have postulated that 
such illustrated reports would be preferred by referring physicians, 
notwithstanding a significant loss of image detail. A survey of these 
referring physicians was carried out to determine whether this 
hypothesis was correct. 

Methods 

This evaluation was performed on sample reports with integrated 
images which were produced on a prototype report/image printing 
system. Most of the production process is slow and labor-intensive 
on this system, but it is capable of producing reports of the same 
high quality that can be realized on the more efficient system which 
would be required for routine clinical use. This long production time 
was not an impediment to the survey respondents, since they were 
dealing only with the paper copies of reports which had been 
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previously produced. The questionnaire was based on partially 
simulated cases, but was otherwise representative of what reports 
from a system in routine clinical use could look like. 

Reports were assembled on a Macintosh Plus computer, using 
PageMaker 3.01 (Aldus) as a page layout program and printed on an 
Apple LaserWriter NT printer, a 300 dot per inch laser printer, over 
the AppleTalk network. Images and text were prepared separately 
on other computers and transported to the Macintosh over a TOPS 
network from an IBM-compatible computer system for integration 
and printing on the Mac. All work could have been done on PCs if 
any of them had been connected to a suitable laser printer. 

The text of the actual radiology report was downloaded from our 
DECrad radiology information system (Digital Equipment 
Corporation) and saved on a magnetic disk. The text was edited to 
remove patient identifiers and extraneous information, to change the 
line length, and to remove line feed characters. 

Images were read from the standard archive tape of our CT9800 
computed tomography scanner (General Electric Medical Systems) 
on a PC equipped with a magnetic tape drive and stored on magnetic 
disk. A program was developed to allow the specification of a 
window width and window level and store the image in the Tag 
Image File Format (TIFF) used by many PC and Mac image 
processing programs, including PageMaker. These images were also 
viewed on the PC to verify that the selected settings displayed the 
significant details of the image adequately. 

The images were then viewed and cropped on a PC using the 
Optimas image processing package (Bioscan, Inc.). This could also 
be used to enhance edges or other aspects of the images, but this was 
not required for these images. 

Initially, the 200 copies of the questionnaire and reports were 
duplicated by an offset printing process. The quality of some of the 
offset printed copies was not adequate so the radiographic images 
were reproduced as laser-printed originals for each questionnaire. 

Opinions 

A one-page data collection instrument was prepared for distribution 
to those categories of clinicians whose patients were regularly 
referred for radiographic examination. Approximately 200 packets 
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contained the questionnaire and eight radiographic reports. These 
reports included versions with and without graphic representation of 
a selected radiograph from the study being reported. When a 
radiographic image appeared on the report, it was annotated with 
respect to key findings. Cases included digitized radiographs of a 
fractured jaw, and a pulmonary carcinoma, as well as computed 
tomograms of septic brain infarcts, and of a pleural fluid collection. 

The clinicians were asked to inspect both sets of reports, and then 
execute and return the questionnaire. In general three themes were 
explored: 

1. Were the reports with images deemed useful by the 
referring physicians; 

2. What were their current practices regarding review of 
radiographs or solicitation of personal consultation 
from the radiologists; 

3. Were their practices likely to be changed significantly by 
the new style reports? 

Results 

A total of 127 questionnaires were returned within two weeks of 
their distribution. Most were filled out by surgeons in various 
specialties, and the balance by physicians or pediatricians. 

The group included both young and older colleagues. Only 20% 
(26/127) had been graduated from medical school for 10 years or 
less, 45% (57/127) twenty years or more, and 35% (44/127) 
comprised the intermediate age group (1 1-19 years). 

The response of the clinicians to the reports was even more 
favorable than expected. Fully 96% either strongly or moderately 
preferred the report with the image to the one without and 98% felt 
the printed images helped them understand the radiology report 
either very much better or some better. 

While 83% said the resolution was adequate or nearly adequate for 
its purpose, 17% thought it needed to be better. None thought the 
relatively poor image quality made the images nearly useless. The 
fact that 94% felt the reports with images would be more helpful 
than the reports alone in explanations to patients is particularly 
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interesting because this method makes the images readily available 
in the clinical care setting. 

While 92% personally review radiographs of patients with positive 
findings in nearly every case or in many cases, only 54% personally 
review the radiographs of patients with negative findings. 

Use of the film library was likely to decrease, with only 1% saying it 
would increase, 43% saying little reduction, and 56% predicting a 
moderate or drastic reduction in their use of the film library. 

Moderate or drastic reduction of telephone traffic with the 
radiologist was predicted by 56% of responses, and 96% felt the 
report with images would be moderately or much more useful than 
present reports in chart review and follow-up clinic visits. 

Conclusions 

Referring physicians thought the reports with images would be 
useful to them in several settings. About half of cases now result in 
the attending clinician having a personal consultation with the 
radiologist, in our teaching hospital setting. Significant reductions in 
telephone and film library traffic are likely when images are 
routinely printed on reports, and efforts to expand our capability to 
do this printing are now in progress. 
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The prompt production and delivery of 
reports is a major problem faced by almost 
every radiology department. The speed with 
which this is accomplished depends on many 
factors, most of which can be optimized with 
adequate personnel and proper management 
techniques. The delay that occurs between 
the transcription of a report and the 
approval by the radiologist is more difficult 
to minimize. 

The purpose of proofreading reports is 
two fold: 1) to correct any typographical 
errors introduced during transcription and, 
2) to ensure that the written report is of 
the correct patient and conveys the 
radiologist's true impressions. In order for 
this exercise to be meaningful, not only must 
the report be read, but the radiologist must 
be able to associate the patients name and 
his diagnostic conclusions with those in the 
report. The more time that elapses between 
dictation and proofing, and the greater the 
number of films dictated in the interim, the 
less likely that the radiologist will be able 
to recall a specific case. This is not a 
problem in low volume areas such as inter- 
ventional radiology or special procedures 
because few cases are done in a day and the 
radiologist is involved with each patient 
over a relatively long period of time. When 
one radiologist reports 50 - 100 cases a 
day, review of reports needs to be frequent 
if it is to be anything other than 
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grammatical correction. 

The problem is compounded in those 
institutions where the radiologists work at 
more than one site. Reports dictated in the 
afternoon are often not returned for 
signature until the following day. If the 
dictating radiologist is assigned elsewhere, 
who signs his reports? Cross-signing by 
other radiologists will improve departmental 
efficiency but certainly skirts the intent of 
the process. 

Electronic signature provides a partial 
solution. It does away with the transport 
and shuffling of paper and, perhaps, makes it 
easier to approve reports. If the different 
sites covered by the radiologists are 
networked, reports can be approved by the 
original radiologist wherever he is working - 
so long as he is not on vacation or away at a 
meeting. Even with electronic signature, 
however, the reports still should be reviewed 
in temporal proximity to dictation so that 
the cases are not forgotten or confused. 
Making corrections can be a problem with 
electronic signature. If the radiologist 
cannot, or will not type, the report must be 
redictated and cycled through the entire 
process one more time. 

A simple solution to all of these 
problems is to have a secretary who can take 
direct dictation, assigned to each 
radiologist when he reports. After each 
examination or so, he can review the report 
and sign it or dictate necessary corrections 
on the spot. Eventually, as voice recognition 
software is refined, it may be possible to 
accomplish the same thing without a 
secretary. However, this technology has not 
arrived as yet. 

Another approach would be to shrink the 
time between dictation of the report and its 
transcription. This can be done, to some 
extent, by using a tank dictation system and 
an adequate number of typists. However, 
because film interpretation occurs 
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sporadically throughout the day, it is not 
practical to set staffing levels according to 
the times of peak activity. In an attempt to 
maximize the efficiency of available 
transcriptionists in our Breast Imaging 
Center, dictation was replaced with a series 
of check sheets where each box represents a 
specific radiologic finding. Each box is 
assigned a code. The radiologist ticks off 
the appropriate boxes and the 
transcriptionist simply enters the codes. A 
relatively simple computer program 
automatically concatenates the appropriate 
phrases to form the report. Unfortunately, 
this did not solve much. Reports dictated in 
the late afternoon are still not available 
until the following day and cross-signing is 
still required. Furthermore, individual 
radiologists may feel unsure about the 
finished reports because entry of a wrong 
digit may profoundly alter the code and its 
meaning. As a result, some radiologists 
require the original check sheet to be 
returned with the typewritten report so that 
the two can be compared. This system 
solves the problem of associating patients 
with their radiologic findings but does not 
achieve the hoped for saving of time. 

Automated reporting systems do exclude 
the transcriptionists from the report 
producing process. They tend to be cost 
effective and can present completed reports 
almost instantaneously, thus alllowing the 
radiologist approval before moving on to the 
next case. Although several comprehensive 
systems have been available for some time, 
they have not met with widespread acceptance. 
This is due in part to the radiologist- 
computer interface but also because of their 
limited vocabulary. In some areas, such as 
chest radiography, the nuances of language 
are required to properly describe film 
findings and to influence the clinician's 
approach to further diagnostic studies and 
treatment of the patient. Other areas, such 
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as mammography, angiocardiography and nuclear 
medicine, deal with a more circumscribed 
group of diagnostic entities and employ a 
rather limited descriptive vocabulary thus 
making them more suitable for computerized 
reporting systems . 

We felt that a graphical computer 
interface would be most acceptable to the 
radiologist and ported the check sheets that 
had been in use for some time to the 
computer. Multiple screens are utilized for 
data input. Using the mouse, the radiologist 
clicks in the appropriate boxes and on pop-up 
diagrams. The report is produced from an 
included dictionary of technical phrases. As 
soon as data entry is completed, the finished 
report appears on the screen and is accepted 
by the radiologist by clicking in a 
password-controlled check box. In our 
department, this report is transmitted 
through an Ethernet network to our management 
system running on a VAX and is printed and 
stored in the same format as all other 
radiology reports . 

The program was originally written in 
HyperCard. However, HyperCard is limited in 
its capabilities as a database. The program 
was then rewritten in a database language 
(4th dimension) so that as each box is 
clicked to generate the report, the 
radiologic findings are transparently entered 
into the relational database. We are able to 
receive pathology reports via a bridge to the 
pathology Ethernet network and, by copying 
and pasting, enter them in the database 
without additional typing. The program is 
being expanded to include calendar function 
for patient scheduling, form letter 
generation to the referring physician and the 
patient as well as reminders for return 
visits, at the appropriate time. 
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Dimensional Medicine Inc. (DMI) of Minnetonka, Minnesota Is poised to 
enter the radiological reporting market with a speech recognition station, 
the DMI-VRS, that is both natural to use and cost effective. This machine 
utilizes two recognition systems. The first, the only speaker-Independent, 
continuous recognizer currently deployed In the radiological arena, Is used 
to drive the only mature, menu-driven, radiological knowledge base, 
Radlex. The second, a natural, speaker-dependent recognition system, Is 
used to provide menu streamlining, freetextlng and full digital dictation. All 
this functionality Is provided at approximately one half the price of 
competitor workstations. This paper describes the capabilities of the DMI- 
VRS workstation and the philosophy used to design it. 

The development of a radiological workstation Is primarily an Interface 
design problem. There Is much to be gained by making radiological reports 
available as soon as possible to those who can benefit from them, however, 
this requires that these reports be entered directly Into a computer. 
Present day computer interfaces are far too cumbersome and sluggish to 
allow direct entry with reasonable effort, especially by those who are not 
experienced computer users. Successful workstations will therefore be 
those that provide intuitive Interfaces that do not detract from the 
productivity of radiologists. 

The majority of modern computer interfaces are unnecessarily 
awkward because they fall to take proper advantage of the years of effort 
humans Invest in language acquisition. To see this we will first lightly delve 
Into the nature of human-human Interaction and use our conclusions to 
expose the shortcomings of modern computer Interfaces. We will then 
discuss how these shortcomings can be overcome using reasonably priced 
present-day technology. The DMI-VRS workstation will be used as an 
example throughout this discussion. Finally we will describe the Interface 
improvements that can be expected in the near future given current speech 
processing research trends. 

if two entitles are to communicate they must both be privy to the same 
language system which will have a general structure of the form depicted In 
figure number 1[1]. The core of this system Is a semantic interpreter 
which allows an entity to extract meaning from the syntactic constructs 
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(sentences, phrases, etc.) generated by and presented to him/her/It during 
Interaction. The lexicon and the grammar define how syntactic constructs 
can be legally formed, i.e., what constitutes correct phrase and sentence 
structure. Finally, the lexical mapping defines how syntactic constructs can 
be physically conveyed. For example, two humans speaking English use 
speech as their lexical mapping, i.e., they assign a different sound to each 
element In the English lexicon, conform to the English lexicon and grammar 
when forming syntactic constructs and semantical ly Interpret these 
syntactic constructs using mechanisms which unfortunately are not yet 
well understood. 



Lexical Mapping 



Lexicon 



Grammer 



C Semantic j 
Interpreter J 



Figure 1: General language structure. 



The acquisition of language is a difficult task for humans. Most of us 
only master one lexicon and grammar, usually English In the USA, and a 
few lexical mappings, usually speech and reading/writing, In our lifetimes. 
Speech Is the mapping most of us use for face to face communication 
because we naturally acquire it and because it allows us to communicate 
rapidly In real time. Read/write mappings are valuable because they allow 
us to record language, however, such mappings are inefficient for face to 
face communication, i.e., consider the difficulty of communicating by 
passing written notes back and forth. Given the large amount of time 
required to master languages It is little wonder that we resist learning 
those that define computer Interfaces. If computer Interfaces are to be 
truly "user friendly" they should incorporate as much of our native 
language skills as is technically pertinent Unfortunately this strategy has 
not been followed in the design of most present day interfaces. 

The languages that define most existing computer Interfaces have two 
major flaws. First, the lexical mappings they utilize are usually 
reading/writing variants, i.e. inefficient message passing mechanisms. 
Second, the lexicons and grammars Involved tend to be highly restrictive 
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usually providing only one legal lexical construct per specific computer 
command, I.e., most commands can only be entered as some exact string of 
characters. Using reading/writing as the lexical mapping slows down 
communication unnecessarily. Attempts to correct this problem, the 
introduction of hot keys, etc., simply replace this problem with another; 
they cause Interfaces to become unsatisfactorily cryptic. We have trouble 
remembering which key Is associated with which command, etc.. Highly 
restrictive grammars make communication frustrating. Humans are bad at 
remembering the exact syntax associated with commands. Human lexicons 
and grammars, on the other hand, are extremely rich. There are usually 
numerous legal syntactic paths available to communicate any specific idea. 
This richness allows us to convert ideas Into appropriate lexical constructs 
with relative ease allowing us to fluently converse on the fly. 

As an example of a current computer interface consider that defined by 
MS-DOS, the operating system utilized by IBM In their original P.C. line. 
The lexical mapping used here Is a visual read/write mapping Involving the 
English/ Arabic alpha-digits and a collection of other symbols such as 
slashes, asterisks, etc. This mapping can be extremely cumbersome when 
entering long commands or In general for those who are not proficient with 
a keyboard. The rigidity of MS-DOS's lexicon and grammar compound Its 
lexical mapping problems. Even seasoned experts constantly find 
themselves having to refer to reference manuals to determine how MS- 
DOS's exacting commands must be entered. 

The prevalent solution to MS-DOS type command line Interfaces Is the 
graphical Interface Introduced by Xerox In 1981 (Xerox STAR) and made 
famous by Apple In their Macintosh line [3]. Such Interfaces use lexical 
mappings which are, In a sense, forms of sign language. The computer 
presents Its operator with a selection of visual symbols, either icons or 
menu entries, from which the user selects some useful sequence. Sign 
language Implements lexical mappings which are as efficient as speech. I.e., 
a signed conversation can be carried out at the same pace as a spoken 
conversation, however, the symbols used in such mappings are not 
naturally acquired and therefore must be learnt. Visual Interfaces do not 
take advantage of the language skills that the majority of us learn as 
children: speech and our native lexicons and grammars. Graphical computer 
interfaces circumvent this shortcoming by visually presenting the user 
with all of his/her possible options eliminating the need for him/her to 
memorize them. The problem with this strategy Is, of course, that only a 
very limited number of symbols can be presented to the user at any one 
time. This Is true because of the limited space available on computer 
presentation surfaces and more importantly, because humans are not 
capable of scanning large lists of symbols with comprehension In a 
reasonably short period of time. Submenus provide some relief for this 
problem but unfortunately tend to slow Interaction down. 

The solution to these Interface shortcomings Is to rely more heavily on 
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our naturally acquired language skills. Computer Interfaces must use speech 
as their primary lexical mapping as well as rich subsets of our native 
lexicons and grammars. Visual aides such as menus should be used to 
complement speech where ever appropriate, however, care must be taken 
to Insure that these aides do not detract from Interface efficiency. This 
overall approach will become increasingly important as the complexity of 
computers Increases In the future. Future technology will strain current 
Interface designs beyond their effectiveness. 

The DMI-VRS workstation was designed with the above philosophy In 
mind. The core of this workstation Is a rich radiological lexicon and 
knowledge base known as Radlex. Radlex is a direct descendant of CLIP [6], 
a lexicon which Is the product of many years of research by radiologists 
working at the Beth Israel Hospital In Boston. The primary lexical mapping 
deployed is natural speech provided by a pair of recognizers that accept 
continuous speech and are backed by a flexible grammar. A digital dictation 
capability is provided for those who do not wish to embrace technology. 

Radlex consists of approximately 13,000 menus of data hierarchically 
organized Into large sets each of which Is related to a specific body area or 
system such as the chest or central nervous system. Every data set 
consists of a master menu which points to numerous submenu sequences 
which contain phrases that are combined to generate reports. Radlex 
automatically selects submenus for the radiologist where ever appropriate, 
guiding him/her through the report generating process. This feature 
eliminates most of the inefficiencies, eluded to earlier, associated with 
submenu selection. 

A typical Radlex session proceeds as follows. The radiologist Is 
provided with patient demographic data, a space where the report being 
generated Is displayed, and the current Radlex menu. When a selection Is 
made from this menu the computer responds by either appending text to the 
report presented In the report display area or by providing the user with a 
new submenu. The end result of this process Is the generation of a concise 
report which clearly describes what alls the patient. 

Radlex Is thus broadly similar to other existing lexicons. The relative 
superiority of one lexicon gver another Is mostly related to its Internal 
structure and how closely this structure reflects everyday reporting 
styles. In particular the grouping of commonly used statements Into a 
pattern which seems natural to the user and which Is repeated wherever 
possible In equivalent situations Is extremely useful. For example, in 
Radlex every limb menu (shoulder, ankle, etc.) provides normal, no 
fracture, fracture and fracture follow up as the first four options. A study 
of 500 limb reports revealed 90% were derived from these four options. 

Just as the modern automobile Is vastly superior to its early ancestors 
because of continuous Incremental design improvements so the present day 
Radlex Is vastly better than its earlier versions due to the same 
Incremental improvements In design and content. Radlex now has copious 
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differential diagnosis lists, optimized sets of qualifying words Inserted 
wherever useful and multiple lists of findings related to numerous topics 
which can be drawn from to justify the conclusions they support. Wherever 
useful, lists of recommendations are available for Inclusion In reports. 
These Improvements endow Radlex with a rich lexicon that Is natural, 
educational and easy to use. 

Menus such as those utilized by Radlex can be viewed as two 
dimensional matrices whose elements can be efficiently selected by stating 
their row and column numbers. This means that by Inputting digits one can 
take complete control over a lexicon like Radlex In an elegantly simple 
manner. This strategy has been utilized In the DMI-VRS to provide a basic 
level of control. Digit Input Is implemented using a speaker Independent, 
highly accurate, continuous digit recognizer. Speaker Independence means 
that this recognizer does not require any training. Anyone can walk up to 
the DMI-VRS and Immediately take complete control over It as they would If 
they had hired a human to do their bidding. No Initial training session 1s 
required. Speaker Independence also means that the DMI-VRS Is Immune to 
the dally variations that naturally occur In the human voice. Even If one 
has a severe cold the DMI-VRS will continue to function well. Continuous 
recognition allows one to enter strings of digits naturally, without pauses 
In between digits, as one would when conversing with other humans. This 
naturalness Is not provided by most other recognizers. 

As stated previously, when designing an Interface that uses a menu 
hierarchy such as that Implemented by Radlex, one must take care to 
prevent Inefficiencies that result from the overuse of submenus. The DMI- 
VRS avoids such Inefficiencies In two ways. First, Radlex's automatic 
menu selection process selects menus for the user where ever possible. 
Second, a second recognizer, a speaker dependent, continuous recognizor 
has been Integrated Into the DMI-VRS to allow user defined streamlining of 
the menu selection process. This recognizer Is constrained by a user 
definable grammar and Is capable of recognizing thousand of user defined 
words. It allows the user to train the DMI-VRS to recognize key words or 
phrases which trigger Radlex to automatically progress through desired 
menu sequences. During normal recognizer operation the user utters 
requests In natural, continuous human speech. Any user defined words that 
exist In this speech are recognized. If these words occur In an order that Is 
legal as specified by the user defined grammar, the computer carries out 
the desired menu sequence. The advantage of this recognizer Is that 1t 
allows the user to define a rich grammar and lexicon combination which 
simplify the command Input process. All Inputs are conveyed using normal 
conversational speech. This recognizer is also used to Implement a 
freetextlng capability which functions In a fashion similar to that described 
above. 

A typical DMI-VRS reporting session proceeds as follows. The 
radiologist Inputs the appropriate accession number using the speaker 
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Independent digit recognizer. He/she then generates his report assisted by 
Radlex and using the digit recognizer. If the radiologist has prereglstered 
streamlining phrases he/she may utter these where appropriate to speed up 
this report generating process. New streamlining phrases may be entered 
Interactively when ever desired. If a freetextlng capability has been 
developed It may also be utilized or amended at any time. If the radiologist 
does not wish to use Radlex he can opt to completely Input his/her report 
using digital dictation. 

Ideally speech processing technology used in computer Interfaces should 
allow humans to converse with their computers as naturally as they do 
with each other. Unfortunately such technology is at least a decade away 
[10]. However, the recent wide acceptance of Hidden Markov Modelling, a 
speech processing technique which produces excellent results, has produced 
some quality, reasonably priced, small vocabulary recognizers such as the 
continuous digit recognizer utilized by the DMI-VRS. Predictions are that, 
due to the embracing of more advanced Markov modelling techniques such 
as Semi-continuous and Continuous Hidden Markov Modelling, high quality, 
continuous, speaker Independent recognizers capable of recognizing 
approximately one thousand words will be commercially available In the 
short term [2,4,5,7,9-10]. Continuous, speaker-Independent recognizers 
with vocabularies containing tens of thousands of words should become 
commercially viable around the turn of the century [10]. True, human-like 
Interfaces should begin to emerge at this time as complementary advances 
In semantic Interpretation techniques also mature. Until this eventuality the 
use of small vocabulary, natural recognizers such as those Implemented In 
the DMI-VRS will provide Interfaces that are far superior to existing 
computer Interfaces. 
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The Clinical Information Advantages, Inc. 
(CIAI) RADstation is a state-of-the-art speech 
recognition system adapted to Radiology. It 
consists of two layers: the Dragon layer, which 
comprises the basic speech recognition software, 
and is based on the Dragon Systems, Inc. speech 
recognition board, functioning in a high 
performance 80386-based IBM compatible computer, 
and the CIAI RADstation software level, which 
customizes the speech recognizer for Radiology* 

The initial clinical trials of the 
RADstation are under way at multiple sites, under 
the aegis of the Radiology Information System 
Consortium (RISC) Speech Recognition Task Force. 
In addition to evaluating the clinical content 
and utility, each of the sites have been looking 
at other aspects also of RADstation integration 
with department operations. At the University of 
Massachusetts Medical Center in Worcester, and at 
the Hospital of the University of Pennsylvania in 
Philadelphia, the RADstation has been interfaced 
to the DECrad Radiology information system. At 
Methodist Hospital in Indianapolis, Indiana, the 
system is being interfaced to the MAXIfile 
system; while at Johns Hopkins University 
Hospital in Baltimore, the interface has been to 
locally developed Radiology information modules. 
Operation at the Massachusetts General Hospital 
in Boston has been as a stand-alone unit at one 
of the satellite facilities. The clinical areas 
exercised at each site have been dependent on the 
clinical interests of the particular radiologists 
using the RADstation at each site. 
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In comparison with earlier speech 
recognition systems, the RADstation is speaker- 
independent, but trainable. This means that the 
average user can go to the RADstation, take the 
microphone and have a large percentage of his 
speech recognized immediately, without having to 
resort to preliminary training. Some of the 
earlier systems required training the system 
fully for each and every word that would be 
spoken by the individual user. The RADstation, 
however, begins with a general speech model able 
to recognize a large percentage of what is 
spoken. Through use, the RADstation becomes 
speaker-dependent, training itself to the 
individual speaker's voice, accent, and 
inflections. Thus, the average user can begin 
using RADstation almost immediately to produce 
reports. A fair amount of easily correctable 
misrecognition or questionable recognition can 
occur at first, using the RADstation; this slows 
the user considerably, during the initial phases. 
This is the learning curve for the machine. The 
learning curve for the user involves becoming 
familiar with the menus, their structure, and 
flow. 

There are two dictation modes available: 
free text dictation and structured dictation. 
Free text dictation is just that: the RADstation 
takes each word as it is spoken and builds the 
text. Structured dictation is the mode where 
trigger phrases are used, macro style, to 
generate complete phrases or phrases with blanks 
that can be filled in. This theoretically 
results in greater efficiency of dictation, 
producing multiple words per utterance instead of 
one. However, there is a significant learning 
curve for structured dictation until the user 
learns the menu structure for this mode. 
Further, there is the problem of not finding the 
phrase that expresses exactly what the user wants 
to say the way he wants to say it. The two 
solutions to this problem are either to accept 
the existing phrasing or to switch to free text 
dictation. 
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The very early clinical experience with 
RADstation has not been without problems. Some 
of these problems are conceptual, some are due to 
performance limitations of the RADstation, while 
others are "bugs". I hope that most of these 
will be addressed before the RADstation ends 
clinical testing. 

In the area of conceptual problems, I would 
include issues of data flow and user interface. 
At the start of a dictating session, the 
individual user logs onto the RADstation, 
identifying himself with a password. However, to 
complete each report after dictating, it is 
currently necessary for the user to again type in 
his password at the keyboard. He then tells the 
RADstation to sign the report. This is an 
unnecessary step. The security identification 
was performed when the user logged on. 

Signing of the report, however, does not 
dispatch it to the receiving systems (except 
perhaps in the stand-alone RADstation) . Because 
the information systems are constantly uploading 
header information for each individual patient 
exam, the RADstation developers elected to hold 
the dispatching of completed reports until the 
user ends his RADstation session. Thus, as 
currently configured, the RADstation does not 
dispatch reports in real-time. 

Because of the word recognition process and 
the necessity of the radiologist to constantly 
monitor which words are being put down by the 
RADstation as dictation progresses, use of the 
RADstation does not occur in the heads-up mode 
where the radiologist is looking at the films as 
he dictates. He has to be concentrating on the 
screen as each individual word appears in free 
text dictation, because subtle misrecognitions 
can always occur. In structured dictation, it is 
necessary to monitor the screen to know where you 
are in the menu structure and what the choices 
are. Most radiologists normally function in the 
heads-up mode. 
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A related point involves the microphone 
system itself. Currently, the microphone is worn 
over the ears with a hand-held on/off switch to 
start and stop dictation. This is cumbersome, 
because putting the microphone on and off is a 
nuisance; one cannot just put it down between 
cases; it has to be removed to use the telephone; 
and the current hand-held dictate switch (a 
standard toggle switch) is not only clumsy, you 
have to remember which way it is thrown. Much 
more desirable and much more in keeping with the 
way radiologists work would be an adaptation of 
a standard dictation system microphone that can 
be picked up from the counter, held to the mouth 
for dictation, and placed down when not in actual 
use, having a push-to-dictate button on the 
microphone and perhaps arrow keys to direct the 
cursor within the screen, if necessary. 

The RADstation has considerable promise of 
being a valuable tool for working radiologists, 
and I hope to have more clinical data available 
at the time of the meeting. 
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Since the establishment of radiology as a 
medical specialty in the United States, 
radiologists have been concerned with providing 
a formal report of their interpretations of 
medical images to referring physicians, and in 
the case of hospital practice to the medical 
records department. When competent medical 
transcr ip tionis ts were readily available and 
technology had no other options to offer, the 
common procedure used to produce a formal 
report was dictation, transcription, review, 
edit (and correction if necessary), signature 
and delivery. Under the best of circumstances, 
this process usually required four hours 
elapsed time from the performance of the 
examination to the availability of the report 
to the referring physician, and it was not 
uncommon for the process in large urban medical 
centers to take as long as forty hours. 
Lengthy delays between the performance of a 
radiographic examination and the availability 
of the report caused further degradation of the 
process by increasing the number of telephone 
calls to radiologists and necessitating visits 
by referring physicians to the radiology 
department to obtain a prompt interpretation. 

For the past twenty five years many 
clinicians, scientists, and workers in fields 
related to the practice of radiology have 
attempted to design and implement systems to 
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facilitate prompt reporting without increasing 
the burden on the radiologist. Such systems 
have ranged from the relatively simple process 
of assigning codes to appropriate pre- 
structured reports, to complex computer-based 
systems requiring interfaces with radiology and 
hospital information systems to provide 
electronic display and/or hard copy reports for 
authorized personnel. In the latter group, the 
early efforts included a system developed by 
Inger Brolin in Sweden that required keyboard 
entry by the radiologist on a Saab terminal. A 
similar approach was developed by Gwilym 
Lodwick and associates at the University of 
Missouri, which utilized an IBM terminal with 
keyboard entry by the radiologist. The RAPORT 
system developed by Richard Mani involved 
"mark-sense" cards that were used by 
radiologists and read by a scanner to produce 
hard copy reports. At Johns Hopkins, a lexicon 
containing the terms and phrases needed to 
report diagnostic interpretations was organized 
for use by radiologists working with touch 
entry devices that displayed the appropriate 
frames for the examination being interpreted. 
The results of this work were made available 
commercially in 1976 in a system called SIREP. 

The most recent efforts to develop a 
system that generates a complete, signed 
interpretation for referring physicians and 
medical records departments involves the 
application of speech recognition technology to 
the radiology reporting functions. With the 
advances in microprocessors and in high-density 
storage devices at relatively low cost, several 
companies have produced commercial versions of 
radiology reporting systems based upon speech 
recognition that are used in a variety of 
clinical settings. While these early efforts 
have certain limitations, particularly with 
regard to speaker dependence, and preclude 
continuous speech input, the prospects for 
improvement are most promising. 
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In order to contribute to the state-of- 
the-art and to provide additional options to 
radiologists concerned with prompt reporting of 
interpretations, the Radiology Information 
System Consortium (RISC) reached an agreement 
with Clinical Information Advantages, Inc. 
(CIAI) to develop a cost-effective system based 
upon speech recognition technology developed by 
Dragon Systems, Inc. The agreement provides 
for the professional requirements and 
functional specifications to be supplied by 
RISC with the software development the 
responsibility of CIAI. Following laboratory 
testing of the initial programs by CIAI, 
clinical testing was begun in several RISC 
member institutions. The term beta testing 
applies to the current activities underway at 
five RISC hospitals where radiologists who have 
contributed to the development of "structured" 
modules dealing with specific body system or 
anatomical/imaging modality modules participate 
by reporting clinical cases. In addition to 
the structured approach, a free text option and 
customized terms and phrases have been 
programmed that may be used independently or in 
conjunction with one another to facilitate the 
reporting process. 




Figure 1. 
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The reporting system, called RAD S t a t i on 1 , 
consists of a 386/33 MHz IBM compatible PC with 
12 MB RAM , a VGA monitor, and a printer which 
varies with the participating institution. It 
can operate as a free standing unit or can be 
interfaced with several radiology or hospital 
information systems. At Johns Hopkins the test 
units are interfaced to the hospital 
information system (HIS). In addition to the 
keyboard, the system is equipped with a mouse 
and a unidirectional headset type microphone. 

Patient identification information on 
cases to be reported (headers) can be entered 
directly into the system or may be obtained 
from the HIS. Headers may be retrieved in a 
variety of ways; by accession number, patient's 
last name or by history number. In addition, 
headers may be sorted by date, exam type or 
alphabetically. At Johns Hopkins, an accession 
number is used to retrieve a specific header. 

Two different methods of dictation may be 
used to generate the report: a menu -driven 
format (structured dictation) and free text. 
The two methods are complementary. Typically, 
structured dictation is used to report normals 
and common findings. This mode provides for the 
introduction by the user of customized phrases 
of personal choice. The free text made is 
frequently used when complex cases are 
encountered or to supplement a report produced 
initially by structured dictation. 

Structured dictation Is organized by 
category (chest, mammography, head CT/MR , etc). 
Each category contains a list of (key phrases) 
with which the user should be able to produce 
most of the reports related to routine 
practice; either as complete reports (normals, 
common abnormals ) or by stringing key phrases 
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together to form a report. Key phrases are 
grouped either anatomically or in selected 
instances pathologically; corresponding to the 
usual radiological approach to the type of 
examination. These groups are called 

"features"; there are usually 8-12 features 
per category, with 10-30 key phrases per 
feature. Certain features (complete reports, 
introduction, clinical history, impression and 
recommendation) remain constant in all 
categories but contain different key phrases 
for the specific category. 

To create a report in structured 
dictation, the user utters the name of the 
desired feature and then the desired key 
phrase. Menu choices are made from within the 
key phrase. A single utterance usually results 
in a five word sentence, frequently much 
greater. For example, saying the word "normal" 
may generate a full page complete report. It 
is this parsimony of utterances combined with 
excellent speech recognition, that makes 
structured dictation a valuable tool. 

The ability to customize one's own key 
phrases is a valuable adjunct to structured 
dictation. The customization feature can be 
entered at any time to create a new key phrase 
or delete an existing one. Thus, one can 
personalize the system with preferred repprts 
and phrases. 

When a case is too complex for structured 
dictation, or whenever the user prefers, the 
free text mode may be used. In free text 
dictation, the user utters one word at time, 
drawing from a vocabulary of up to 30,000 
words. The speed and accuracy of free text 
dictation has yet to be established, but 
preliminary results are highly favorable. At 
Johns Hopkins, several users dictate almost 
exclusively in the free text mode, attesting to 
its performance. 
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Within the system, training occurs in 
three different ways. It is important to note 
that no speech recognition training is required 
before use of the system is attempted. As soon 
as a user is added to the system reporting of 
cases may begin. Training occurs "on the fly" 
and as a user gains experience speech 
recognition improves. In the free text mode, 
if a word is mi s recognized , the user types the 
first few letters of the correct word until it 
appears on the screen and then continues 
dictating. This "on the fly" training allows 
the system to relate syntax to recognition, as 
well as the speech pattern for a specific word. 
Thus, free text speech training occurs every 
time free text is used. 

In structured dictation speech recognition 
training occurs only when the user initiates 
it, by saying "scratch that" to delete a 
misrecogni t ion , then saying "train this word" 
to activate the training mode. Again, the 
first few letters of the correct word are typed 
until it appears. Then a function key 

corresponding to the correct word is pressed 
and the system trains that word. In the third 
approach, a user can invoke a feature called 
"retrain words" that allows specific words to 
be trained in structured dictation. This 
function is particularly helpful when the user 
is experiencing misrecogni tion problems between 
s imilar words . 

Once a report is complete the user can 
verify its accuracy on the screen and then by 
saying "end dictation" may "deliver" that 
report. A variety of levels of security exist 
(preliminary reports for residents, final 
reports for attendings) depending on the local 
use of the unit. Hard copy is then generated 
locally and electronic copies are transmitted 
to the HIS for distribution. 
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As of this writing (April 1990) the 
results of the beta test are not available. 
Those that are documented will be presented at 
the Session on Reporting at the 10th Conference 
on Computer Applications in Radiology. 
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Introduction 

A number of digital radiography systems are recently available 
for clinical studies. In particular, the clinical use of Computed 
Radiography(CR) system using photostimulable phosphor 
imaging plate has been increasing in diagnostic radiology as a 
new modality which has potential to replace the conventional 
screen-film system. 1 - 2 

Furthermore, one of the future potentials of CR system is 
likely to be in the fields of Computer-Aided-Diagnosis(CAD). 
The American Cancer Society has recommended the use of 
mammography for screening of women with annual 
examinations over the age of 50, so that mammography may 
eventually constitute one of the highest volume of x-ray 
examinations interpreted by radiologists. 3 For this reason, 
evaluation of breast tumors in mammograms are recently one 
of the most important topics in diagnostic radiology. If 
computerized quantitative methods can be developed which 
provide the reliable measures to discriminate between 
malignant and benign tumors as an objective assessment, it is 
likely that the accuracy of radiological interpretation would 
greatly increase. We are preliminarily investigating an 
application of CAD involving an automated method for 
calculating physical characteristic measures of breast tumors 
in CR mammograms. 
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Materials and Methods 

Digital image data of mammograms are directly acquired by 
means of Fuji Computed Radiography system. After the x-ray 
exposure, the photostimulable imaging plate are scanned by 
laser beam and the detected photostimulable luminescence 
radiation are converted to digital image data with a 0.1 mm pixel 
size and 10 bit density resolution. The image data are 
transferred to a SUN3/260 computer(Sun Microsystems, 
Mountain View, Calif.) through the dedicated optical-disk filing 
system for FCR system. 

Some features of malignant and benign breast tumors are 
summarized in Table 1. The basic principle of our method in 
order to convert these subjective features into objective 
physical measures is shown in Figure 1. A square ROI 
including a region of breast tumor is selected by a manual 
method according to the tumor size. 

The image data variation observed in a selected ROI consists 
of not only tumor signals but also structured background and 
noise components. Therefore, bandpass filters are applied to 
the ROI in order to remove the structured background trends 
and the noise effects due to mammary glands and x-ray 
quanta, respectively. Next, a polar coordinate transform is 
employed to the ROI for the purpose of effectively extracting 
the data variation observed along tumor surrounding. 

The power spectrum after the polar coordinate transform are 
obtained by a two-dimensional fast Fourier transform. Finally, 
the root-mean-square(RMS) value and the 2nd moment of the 
power spectrum in Rs direction(Rs') and 6s direction(9s') are 
calculated in order to discriminate between benign and 
malignant tumors. RMS value quantifies the magnitude of 
tumor internal variation. Rs' value and Gs' value, which 
represent the 2nd moment of the power spectrum in each 
direction, quantify the fineness of tumor internal variation and 
tumor surrounding variation, respectively. 
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Results 

The analytical results of our method are shown in Figure 2., 
which are obtained from the typical malignant and benign 
breast tumors in CR mammograms. Malignant tumor pattern 
has a higher frequency component in both directions of power 
spectrum and a larger RMS values compared with benign 
tumor pattern. As a preliminary study, the results of 29 images 
containing 14 malignant tumors and 15 benign tumors are 
shown in Figure 3. It is indicated that RMS variation and the 
2nd moment of the power spectrum of malignant tumors are 
clearly different from those of benign tumors. 

Discussion and Conclusion 

The elimination of background trends and noise effects is a 
very important step prior to the computation of those physical 
characteristic measures, because it removes the effect of the 
undesired data variations and thus makes it possible to isolate 
the finely detailed pattern due to breast tumors from the overall 
breast density variations. Moreover, a polar coordinate 
transform is very effective to extract the data variation obtained 
along tumor surrounding, so that the combination of the RMS 

value and the 2nd moment of power spectrum in 9s direction 
more clearly discriminates between malignant tumors and 
benign tumors. 

We have developed an automated method for determining 
physical characteristics of breast tumors in terms of the RMS 
value and the 2nd moment of power spectrum of the breast 
tumor patterns, which represent the magnitude and fineness of 
internal structure of breast tumors, respectively. We conclude 
that these physical characteristic measures calculated from CR 
mammograms may be useful to radiologists in their differential 
diagnosis of breast tumors. 
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Table 1. Features of malignant and benign breast tumors 
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Figure 1. Basic principle of the automated method for calculating an 
objective physical measures of breast tumors in CR mammograms. 
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Figure 2. Analytical results by means of our method. (a)Result 
obtained from the typical benign breast tumor. (b)Result obtained from 
the typical malignant breast tumor. 
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Figure 3. Preliminary results of 29 images containing 14 malignant 
tumors and 15 benign tumors. It is indicated that RMS variation and 
the second moment of the power spectrum of malignant tumors are 
clearly different from those of benign tumors. 
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Abstract 

This paper reviews the application of 
artificial intelligence (AI) in diagnostic 
imaging. All current and anticipated 
applications are briefly discussed* A basic 
understanding of radiology and artificial 
intelligence is assumed. 

Introduction 

Applications of AI in diagnostic imaging can 
be divided into three categories: 1) 
Applications that involve the acquisition, 
display, and analysis of images; 2) Applications 
that attempt to analyze, critique, and improve 
the cognitive skills of radiologists; and 3) 
Miscellaneous applications. See Table I. 

In this report, a distinction is made between 
advice programs ( AP ) and front-end programs 
(FEP). Advice programs include expert systems 
(ES), neural nets (NN), statistical programs, 
and others which use ordered lists of primitive 
features as input; output is advice (1), Front- 
end programs use image or numerical data as 
input and provide a list of primitive features 
as output. This output is suitable input to an 
AP . Sometimes a human acts as the FEP providing 
the input data in an interactive session with a 
computer, and sometimes a program acts as FEP. 

In the future, many of the progr ams discussed 
i n this article will be components ot radio logy 
work s tat ionsT These systems, still in the 
developmental stage, will include picture 
archiving and storage systems, radiology 
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inf ormation systems , and computerize d advice and 
critiquing systems . There will E~e user 
interfaces designed to ease: 1) the acquisition 
of the radiographic report, 2) image 
manipulation, and 3) interaction with the other 
components of the system (2). 

On occasion, it is difficult to make a 
distinction between those programs which use 
artificial intelligence methodologies and those 
that use related but non-artificial intelligence 
techniques, For the purposes of this 
discussion, all AI programs or closely related 
programs that serve similar functions are 
discussed regardless of methodology, I regret 
any omissions of reports of systems of which I 
am unaware. 

Imag ing Applications 

Test Ordering 

As imaging technology becomes more complex, 
it is becoming increasin gly HiffimiU. fnr non- 
radiologists to know the appropriate sequence of 
radiologic test(s) to order for a particular _ 
patient . The PHOENIX Radiology Consultant (3-5) 
is an expert system that offers advice to 
physicians ordering diagnostic imaging 
procedures. This system provides an 
"appropriate , sequential work-up for a variety 
of common clinical problems' 1 . Another AI 
approach has been described (6)* There have 
been non-AI computerized approaches to the 
problem of test ordering, (7-9) as well as 
several handbooks published recently (10-11). 

Test Performance Advice 

Computed tomographic and magnetic resonance 
scanners are complex devices. Many variables 
must be determined before performing an exam. 
There is a report of an AI system to set up 
magnetic resonance protocols. (12) 

Machine Vision 

The perception of features (in an image) by 



168 



computers is termed machine vision. Since many 
of the images in radiology are digital, the 
distinction between machine vision systems and 
pattern recognition programs is not clearcut . 
Both types of programs are considered to be FEP . 
Their outputs can serve as input to an AP . 

Expert Systems 

The analysis of image features to suggest a 
diagnosis has been performed by several expert 
systems (13-14). In some of these systems, the 
radiologist acts as the front-end program (15- 
21). In others, the input is the raw image 
data, and the front-end program and expert 
system are internal components of the system 
(22-27), These are summarized in Table II. 
Most of these programs are not available 
commercially. However, there was a brief notice 
describing the View Station 500 by Streometrix, 
a system that assists radiologists in 
interpreting mamograms (28). Claims were made 
that clinical studies show a 95 percent accuracy 
rate . 

Neural Nets 

Neural Nets have recently generated renewed 
interest* Along with expert systems and 
statistical techniques, these systems are viewed 
as a type of computerized advice system which 
may be part of radiology work stations. I am 
unaware of any reports of neural networks in any 
Radiological Advice Program. 

Quality Control 

AI programs can play a role in quality 
control. Todd-Pokropek has described a system 
which uses an expert system as part of a quality 
assurance computer program in Nuclear Medicine 
(29), This uses input data from flood field 
images and other quality control procedures. A 
front end program transforms this data into 
primitives which act as input to an expert 
system. Another system has been described to 
aid repair personnel in repairing computerized 
tomographic scanners (30). 
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Cognitive Applications of AI 

Analysis of Radiographic Decision Making ^ — — 

When a radiologist examines a X-ray and 
interacts with an expert system supplying the 
findings on the film, he is serving as a "front- 
end pro gram ' 1 . It is believed that it Ts 
possible to measure aspects of his performance 
as a front-end program* Likewise , asking the 
radiologist his diagnosis after examining the 
film, one would be able to measure his 
performance as an "advice program' 1 . While this 
separation is artificial and not necessarily how 
radiologists perform their tasks on a daily 
basis, it is a way to measure the relative 
contributions of perception (i.e. , as FEP) and 
reasoning (i.e., as AP ) to the radiological 
decision making process (31). This has been 
studied on a limited basis (21). In order to 
perform this analysis, a validated expert system 
and test case database are required (32). 

Critiquing Radiologists 

Swett, et . al . , have demonstrated the utility 
of expert systems for critiquing the diagnoses 
of radiologists (33). In this system, the 
radiologist interacts with the computer by 
supplying history, pertinent findings on the 
image, and a diagnosis. The expert system then 
analyzes these data and critiques the proposed 
diagnosis. This system has been extended so 
that in addition to critiquing the performance 
of the radiologist, it makes available 
illustrative radiographic examples both of the 
findings and of the criticisms that the computer 
has made of the radiologist's actions. These 
close the feedback loop (34). 

Teaching and Testing 

Two unexplored applications of AI in 
Radiology are teaching and testing. One might 
argue that the approach of Swett, et.al., 
discussed above is a form of teaching and may 
result in increased knowledge of the user. 



170 



However, there is also a role for drill and 
practice in radiological education* The 
database of cases necessary for an expert 
system's development can be used as examples for 
training. In a similar fashion, this same set 
of cases could be used to compare a new 
observer's responses with the database of "true" 
findings and could be used to assess the skill 
of radiologists. 

Miscellaneous Uses of Artificial Intelligence 
Robotics 

Robots were introduced into the medical 
laboratory in the early 1980 ' s (35). 
Subsequently, robots have been used in the 
preparation of radiopharmaceuticals. This 
allows volatile chemicals to be used and 
critical reactions to be performed without 
excess radiation exposure to humans (36-28). 

f> Natural Language Understanding 

The principal application of natural 
language understanding systems in Radiology is 
automated report generation. Kurtzweil and 
others have demonstrated automated speech 
understanding systems that use vocal input and 
produce a typed report. The Kurtzweil system is 
currently capable of understanding 5000 words 
from several different users (39-40), It is 
possible to FAX these reports to the referring 
physician's office as soon as signed (41), 
Other systems are under development (42-43), It 
is also possible to use natural language 
understanding systems to allow vocal control of 
devices (44). The availability of PC-based 
speech recognitions systems should allow 
continued development of these applications 



(34) . 
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Table I* 

Outline of the applications of Artificial 
Intelligence in Radiology used in this paper. 

A. Imaging applications 
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Determining which test to order 


2. 
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3. 


Machine vision 
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Expert Systems 
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Neural nets 
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Cognitive applications 
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Analyzing radiological decision 
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report generation 
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Table II 



A List of Expert Systems for Radiologic 
Diagnosis . 
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FEP = Front-end program, AP = Advice Program, 
REF = Bibliography entry 
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ABSTRACT 



The Radiology Image Interpretation System 
(RIIS) is an expert system used to study 
applications of expert systems to radiographic 
image interpretation. The system takes a 
description of the radiographic abnormalities 
of a focal skeletal lesion and, through 
interactive questioning of the physician, 
produces a list of the most likely diagnoses, 
RIIS uses both a rule based approach and a 
match and score technique to produce the list 
of the most likely diagnoses. The diagnoses 
are ranked according to the relative 
likelihoods produced using the match and score 
technique. An observer-performance study was 
done comparing the efficacy of RIIS on a known 
set of abnormal and normal cases. Modified 
receiver operating characteristic (ROC) curves 
for inexperienced residents, experienced 
residents and RIIS were generated using 
radiographs of known focal bone abnormalities. 
On the average, the RIIS system was less 
accurate when compared with experienced and 
inexperienced residents. This difference was 
not significant for inexperienced residents* 
RIIS performed better than inexperienced 
residents when used by experienced residents. 
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INTRODUCTION 

Expert systems have been applied to a wide 
variety of medical problems [1]. Several 
investigators have been studying the 
application of expert systems to radiographic 
image interpretation and consultation [2,3]. 
One problem with the application of expert 
systems to image interpretation is that most 
of the present systems rely on input into the 
system in the form of findings on a 
radiograph. If the observer is inexperienced, 
the input can then be inaccurate leading to 
inaccurate differential diagnoses [3]. 

ROC curves are used to study the accuracy of 
signal detection. They have been used in 
evaluating the accuracy of many diagnostic 
systems [4]. The application of observer- 
performance studies to expert systems has been 
limited [5]. ROC curves allow comparison of 
observers and modalities over the complete 
range of false positive values and are 
therefore useful in evaluating expert systems. 



RIIS was developed to study the application of 
expert systems to radiographic image 
interpretation. RIIS interactively questions 
the user about the findings on a radiograph. 
The sequence of questions and which questions 
are asked depends upon answers to previous 
questions. RIIS makes several intermediate 
conclusions using a rule based approach. 
These intermediate conclusions are used to 
classify a broad category of differential 
diagnoses. Differential diagnoses in the most 
likely category or categories are then 
considered for the final differential 
diagnosis. 

A match and score approach using relative 
frequencies and relative predictive values of 
specific findings for a specific diseases is 
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used to rate the relatively likelihood of each 
disease. This approach is modeled after the 
Internist program [6]. Diseases are then put 
in order according to their relative 
likelihoods. At present the expert system is 
limited to focal bone abnormalities. 

This study evaluated RIIS using an observer- 
performance study with radiology residents of 
different experience. 



METHODS A*m MATTCPTAT.fi 

Fifty seven cases of known focal bone lesions 
and ten normal cases were reproduced as black 
and white prints. The reproductions were used 
as test cases for RIIS and the radiology 
residents. The abnormal cases were selected 
from the Cleveland Clinic Teaching File and 
each had a known pathological diagnosis. The 
normals were selected from routine clinical 
exams considered normal by a musculoskeletal 
radiologist. 

Four residents or fellows with at least three 
years of radiology training were the 
experienced residents. Four residents with 
less than one year of training were the 
inexperienced residents. The users first 
select their own differential diagnosis of a 
case from a list of diagnostic possibilities 
including normal. Each diagnosis could be 
selected with a confidence level from 0 to 100 
with increments of 10 (0- definitely not the 
diagnosis, 100- definitely the diagnosis) . 
RIIS was then used by the same resident to 
produce a differential diagnosis for the same 
case. The resident's diagnoses as well as 
RIIS 1 diagnoses were automatically saved. 

ROC curves were constructed by calculating the 
true positive rates (true positive 
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number/ (true positive number + false negative 
number)) and false positive rates (false 
positive number/ (false positive number + true 
negative number)) for RIIS and the residents. 
The resident's and RIIS 1 decisions were 
clarified in four alternatives: 

1. true positive decision, 
decision is the true 
positive or pathological 
diagnosis; 

2. false positive decision, 
decision is not the 
pathological diagnosis; 

3. true negative decision, 
decision that is not the 
pathological diagnosis is 
correctly avoided; 

4. false negative decision, 
the pathological 
diagnosis is not made. 

The true positive rates at false positive 
rates of 0.02, 0.05, 0.1, 0.2 and 0.5 were 
calculated from each specific ROC curve. 
These true positive rates were then compared. 



RESULTS 

The average difference between the resident's 
true positive rates and RIIS' true positive 
rates with that resident are presented below. 
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Average Differences Between RIIS and 
Residents at Different False Positive Rates 



False 

Positive 

Rate 



Average 
Difference 
(Computer 
Experi- 
enced 
Resident) 



Average 
Difference 
(Computer 
Inexperi- 
enced 
Resident) 



0.02 



-0.24 
S.D. 0.09 



-0.12 
S.D. 0.07 



0.05 



-0.1 
S.D, 0,10 



-0.12 
S.D. 0.12 



0.10 



-0.13 
S.D. 0.10 



-0.05 
S.D. 0.12 



0.20 



-0.11 
S.D. 0.06 



-0.02 
S.D. 0.12 



0.50 



-0.07 
S.D. 0.04 



-0.02 
S.D. 0.07 



On the average RIIS did not perform as well as 
experienced or inexperienced residents. This 
difference between the user and RIIS was 
marginally significant for experienced 
residents at a low false positive rate, but 
was not significant for inexperienced 
residents. RIIS* performance with experienced 
residents was better than the performance of 
inexperienced residents alone. Experienced 
residents also outperformed inexperienced 
residents. 



DISCUSSION 



Evaluation of medical expert systems is 
difficult [7]. Expert systems must be 
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evaluated in order to judge their 
effectiveness in the medical environment. 
This study demonstrated that RIIS, an expert 
image interpretation system, on the average 
performed worse than its users for diagnoses 
of focal bone lesions. This difference was 
only marginally significant for experienced 
residents at low false positive rates and not 
significant for inexperienced residents. 

The implications of these results are that 
with the present RIIS system, an average 
user's performance would not be improved by 
help from the expert system. It should be 
noted that in some specific instances RIIS 
outperformed the user and in these instances 
RIIS might improve the user's performance. 

The fact that RIIS was capable of better 
performance than inexperienced residents shows 
that RIIS could improve the performance of 
inexperienced users if an "accurate" 
description could be obtained from them* 
Performance of the RIIS system could also be 
improved by adjusting the knowledge base. The 
present knowledge base has several known 
inaccuracies which need to be corrected. 

Future expert image interpretation systems 
need to account for variations in users 
ability to "accurately" describe the important 
findings. This could be accomplished by 
direct input of the image, modification of the 
expert system by experience level and user 
specific knowledge bases. If "accurate 
descriptions" could be obtained from 
inexperienced residents, RIIS would be capable 
of improving their performance in differential 
diagnosis of many focal bone lesions. 
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Introduction 

Artificial intelligence in medicine is a field of research that has seen a fair 
amount of activity over the last twenty years. Computer-based expert systems have 
been designed in many domains of medicine. Yet very few of these systems are in 
routine clinical use. What is the reason for this almost universal rejection by the 
medical community? Arc computer expert systems just not good enough? Obviously, 
this is not so, because systems such as Internist have been shown to perform at levels 
close to the best experts in the field (Miller et al, 1982). There must be some other 
factor that conventional expert systems lack. We believe that the missing factor is 
simply case of use. Most expert systems have been created to prove the point that 
computers can capture expertise in restricted domains - they were never designed to 
fit into the busy environment of the clinic. From one point of of view, the focus of 
medical expert system designers on the representation and manipulation of medical 
knowledge was necessary, because that is a formidable task in itself. Now that some 
inroads have been made in that direction, however, it is time to design expert systems 
so that they address the problems of interfacing with busy physicians dirccUy. This 
paper discusses some of the design considerations that have to be incorporated into an 
expert system from the outset if it is to become easy enough to use routinely in a 
clinical setting. This is based on our experiences in designing the IMAGE/ICON 
system which is a prototype expert system that provides both text as well as image 
output as an aid to radiologic differential diagnosis (Swett et al, 1989). 

The domain of radiology itself presents a great challenge for expert system 
design. Traditional expert systems in medicine provide verbal material such as 
diagnostic criteria, lists of differential diagnosis, additional clinical or radiologic 
features to search for to better characterize an abnormality, or distinguishing features 
between several possible diagnostic candidates. Such verbal explanations can 
strengthen a physician's cognitive analysisof a case. In a visually oriented discipline 
like radiology , however, a great deal of reasoning occurs at a primarily visual or 
pattern recognition level. It is difficult for computers to help in this because optical 
pattern recognition by computer is still at a very rudimentary stage. A central design 
philosophy in our exploration of knowledge-based systems in radiology has been to 
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find ways for computers to complement human native intellectual skills by exploiting 
their unique ability to organize and recall information. In IMAGE/ICON therefore, 
we explore the possibility that a computer expert system might also select and display 
relevant images from a potentially large database of proven cases in a manner that 
would help radiologists to reason visually, In this paper we focus primarily on how 
we have attempted to achieve intelligent image retrieval in the IMAGE/ICON system. 

In order to integrate intelligent image display smoothly into a reading room 
setting it is necessary to 1) Allow the radiologist to keep focus on the case at hand, 
yet anticipate his or her diagnostic needs; 2) Retrieve a set of images that meet 
those needs in a focusscd way. 

Anticipation of Radiologist's Diagnostic Needs 

The idea that images can be useful to a radiologist is based on the recogni- 
tion experience that most radiologists will find it very easy to identify with: "Oh, yes! 
I remember a case which had a lesion similar to this, This is obviously disease X", 
Sometimes this recognition experience is as clear-cut as described above. More often, 
however, the experience is somewhat more fuzzy: "I do recall that similar lesions are 
seen in diseases Y and Z. But the border of this lesion is somewhat unclear...". At 
such times the radiologist has to rely on imperfect human recall. If a computer system 
could, at this very moment, produce proven images of both the diseases in question 
comparison would be aided tremendously. 

Two points should be noted about the above scenario. First, it is essential 
that the radiologist be allowed to focus on the case at hand. Typically, the radiologist 
would be dictating the case report for transcription. To have the radiologist take time 
out to describe the findings via lengthy keyboard interaction would be a distraction 
and waste precious time, thus making the value of any benefit of the expert system 
advise quite questionable. Secondly, the exact nature of the images that it might 
benefit a radiologist to see will vary depending on the specific intricacies of the 
problem domain as well as during different stages of the case analysis. A successful 
radiologic expert system must be able to deal with these problems. 

The only solution to the first problem seems to be to combine the expert 
system with a computer-based reporting system. In that way all the information that 
is being described about the case would already be known to the expert system. This 
is easy enough to do in those few areas of radiology where check-off forms are used, 
as for example in mammography screening at some centers. Computer screens can 
easily be designed to mimic paper forms and can even improve substantially on them. 
IMAGE/ICON uses Apple Macintosh™ style dialog boxes and menus for entry of 
cases if this mode of entry is acceptable. In most areas of radiology, however, voice 
reporting is still extensively or almost exclusively used. We have interfaced IMAGE/ 
ICON to a commercial radiology voice reporting system (VoiceRAD, developed by 
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Kurzweil Applied Intelligence Inc.) that uses a trigger-phrase lexicon to generate 
radiographic reports* This is very exciting because it allows IMAGE/ICON to work 
'in the background* as the case report is being dictated, coming up with its text and 
image output whenever required, without the radiologist having to anything different 
from what he or she normally does. The fact that the cases are dictated into the 
system using discrete trigger words and phrases, makes it easy to translate them both 
into the complete sentences needed in the report, as well as the 'tuple* representation 
required internally by the expert system. Thus, if the radiologist says "Mass small 
high density location sub-cutaneous", the voice system will enter the following 
sentences into the report: "There is a small high-density mass. The location of the 
mass is sub-cutaneous." At the same time, the case data is piped into IMAGE/ICON 
in the form of name-attribute-value tuples such as (mass present yes), (mass size 
small), (mass location subcutaneous), etc. After the radiologist has finished dictating 
the report, which includes the tentative diagnosis, he or she can say "Analyze". This 
signals the expert system to which come up with a text critique and a set of images 
relevant to the case. At this point, the radiologist can examine the critique and the 
images in detail, after which he has the option of changing his diagnosis before the 
final report is printed. 

Voice recognition technology is improving steadily, and systems that can 
understand continuous speech may start becoming available a few years from now. 
These would undoubtedly provide the best possible expert system interfaces as they 
would truly allow most radiologists to report cases as usual, and get expert-system 
advise without any further effort The integration of continuous speech with expert 
system advise would seem to require sophisticated natural language parsing in order 
to translate the report into the internal symbolic form required by the expert system. 
This problem might be alleviated by the fact that some syntactic parsing would 
already be done by the voice recognition system(this might be a sine qua non for 
continuous speech recognition), so that it would possible to get some kind of proc- 
essed output from the voice recognition box. Also our preliminary studies show that 
in constrained domains such as medical reporting it is possible to make sense of 
natural language text by judicious search for keywords plus identifying all possible 
expressions that signify negatives. 

The second aspect of getting an expert system to fit into the reading room 
environment is to anticipate what images would be most useful to the radiologist at 
different stages of the diagnostic process. As mentioned above, these would differ 
from domain to domain and require critical study of the area in question. In the area 
of mammography (one of the domains for which an IMAGE/ICON module has been 
designed) these are some of the questions that the radiologist has to wrestle with: 

1 ) Is this film technically well done? Do I need a repeat? 

2) Would a different view or a compression view be useful in this case? 

3) Is this a lesion or merely a normal variant? 

4) Is the difference between the current examination and the previous one significant? 
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5) I think this is entity X. Does entity X cause findings similar to what I am seeing? 

6) What is the range of appearances of entity X? 

7) What are all the benign things that resemble this finding? 

8) What are all the malignant things that resemble this finding? 

The list is obviously incomplete. In most cases the answer to most of these 
questions would be obvious. But when it is not, looking at a group of images that 
illustrate the point in question could be a great help. In IMAGE/ICON we have tried 
to this by using the Axis display. 

Axis Display 

The basic unit of image display in IMGE/ICON is the image axis. An axis 
consists of groups of images that are similar in well-defined ways, and that are 
ordered according to well-defined criteria. From the user's point of view, an axis is 
nothing but a set of images that provide a focussed answer (in the form of images of 
proven cases) to a specific spontaneous query. Thus, in the case of query 3 above, it 
would be helpful to see a group of images that presents a range of normal variants 
that can be confused with a given finding , followed by images showing actual 
examples of the troublesome finding ordered from most subtle to most obvious. It 
would then be much easier to call the finding one way or the other. 

An important aspect of an image axis is that the key features of the similar- 
ity are generalized from the case at hand. Thus if a mass is described as a finding in 
the current case, the above example would show subtle examples of masses; if a 
cluster of microcalcifications is described as the finding in another case, subtle 
examples of clustered microcalcifications would be shown in the same axis. 

An example of the axis display is shown in figure 1 . Each of the 5 horizontal 
rows of small images on the right side of the screen represents a single axis. The first 
panel gives the title of the axis, and the number of images it contains. This is 
followed by the images themselves, ordered in whatever manner is adequate for that 
particular axis. The axis images can be scrolled laterally in order to examine the 
whole axis. Also, if any of the small images is found to be similar to the case at hand, 
clicking on it with the mouse enables the user to examine it at full resolution. 

In order to dynamically generate such axes, it is necessary for the system to 
know exactly what each image contains. For this purpose, we have developed a 
prototype image description language. Information about every image is stored in the 
form of a coded database of findings. Every finding on the radiograph is assigned its 
own set of unique descriptors that can take on different values and are associated with 
scalars. Thus "size of mass" would be a field-descriptor for a mass which would 
contain a numeric value representing the size of a mass. Similarly, one field descrip- 
tor for a calcification would be "type of clustering", where the value could be local- 
ized or diffuse. Every sub-finding of a finding can be represented (when warranted) 
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Figure 1. Axis Display 

as a finding itself, defining its own specific descriptors. Thus, if in a certain radio- 
graph it is thought that the border of a mass should be described in very great detail, 
that can be assigned the status of a finding in its own right 

The field descriptors cover various different aspects of a finding including 
size, shape, location, type, typicality, conspicuousness, geographical and etiological 
relationship to associated findings, dynamic temporal behavior, spatial extent in other 
views, etc., in addition to descriptors unique to the finding. 

Every finding attribute field includes a 5-point scalar which gives the degree 
or intensity of whatever scale can be naturally applied to the attribute in question. The 
scalar means different things for different finding attributes. For example, to describe 
the shape homogeneity of microcalcifications the scale would go from 1 for inhomo- 
geneous (having very different shapes) invasive to 5 for extremely homogeneous (all 
having similar shapes). On the other hand for temporal behavior involving mass 
growth, 1 would mean very slow growing and 5 would mean very rapidly growing. 
The inclusion of such a scalar enables the coding of extremely precise information, 
which is required for very sophisticated querying of the image database, that is 
specifically used in axis-based retrieval. 

Internally in IMAGE/ICON, an axis consists of a Boolean set of prioritized 
conditions that constrain a search, and another set of prioritized conditions that 
organize and order the images found in the search. Any of the fields in the image 
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description may be used in the conditions. The constraining conditions are usually 
derived by generalization from the case being described currently so that they are 
relevant 

The axes in the mammography module of IMAGE/ICON are derived from 
the kinds of questions shown above. In addition two general purpose axes are: 1) The 
Best-match Axis, which orders images in the database depending on how well they 
match the current case; and 2) The User-defined Axis, where the user can specify 
appropriate search and ordering criteria. 

Conclusion 

If medical expert systems are to be used routinely, considerable care has to 
go not only into the design of their knowledge bases, but also into their interfaces and 
into how they will fit into the clinical environment The success of medical expert 
systems is not only going to be dependent on what they have to say, but how they say 
it and whether they say it at the right time. 
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Summary 

This paper deals with continuous modelling for 3D medical image analysis. This 
reconstruction method is initially more complex to implement than the Cuberille 
method but it takes advantage of algorithmic geometry technics which provide powerful 
means for 3D image processing. High quality 3D image rendering is performed and a 
new 3D boundary tracking method based on differential properties has been developed. 

1 INTRODUCTION 

3D images are widely used in medical imaging. They are generally constructed 
from parallel serial section images obtained from Computed Tomography or Magnetic 
Resonance Imaging. 

Our aim is to provide the clinician with new computer graphic tools which help 
him in three-dimensional image analysis for diagnostic and therapeutic purposes. A 
3D medical image analysis system has to suit some of the following specifications : 

- to improve the understanding of the shapes of the studied objects and then to 
help a surgical planning, 

- to enable to quantify the object by morphological parameters such as the 
volume, 

- to be able to compare the images produced by various imaging systems (CT, 
MRI, X-ray pictures...) and to match these multimodality information, 

- to perform an accurate estimation of the relative position of objects. That point 
is essential to guide a surgical intervention, for instance when introducing a needle 
into the brain or when focusing radiotherapy. 

This paper presents our choices to perform this analysis system and our current 
developments. In each section the state of the art in the concerned area will first be 
presented. 

2 VOLUME MODELLING 

In most present 3D imaging applications, the 3D scene is represented by a 3D 
array of volume elements named voxels. This "Cuberille" modelling [HER 88] 
requires either an isotropic resolution of initial data (and then many slices) or to 
compute intermediate slices (by interpolating the original cross-sections for instance). 
Moreover, volume rendering sometimes needs to compute surface gradients which 
have to be approximated [HOH 86]. 

The continuous modelling allows to get around these problems. Instead of using 
discrete elements to model data, we use 3D continuous elements which are made of 
tri-cubic B-spline functions. Thus the density f is defined in every point (x,y,z) of the 
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volume and is written as: 

f(x,y,z) = £ CCijk* B(x-i) * B(y-j) * B(z-k) (Eq. 1) 

where B is the basis B-spline function: 

B(t) = t3 + - 4*(t-j) 3 + + 6*(t4> 3 + - 4*(t- J)3 + 

Wenote: Y+= Y ifY>0 

0 elsewhere 

This density function f has to approximate the initial slices. This 3D approximation 
problem is easily solved because it can be broken up into 3 one-dimensional 
problems. We chose to smooth the initial data in each slice and to interpolate between 
slices [CIN 87]. The major advantage of this model is to allow the computation of 
density and its derivatives anywhere in the volume. Thus ray-tracing volume rendering 
is available for a low number of slices (often corresponding to the clinical use) and 
provides better quality images achieving maximum sharpness without introducing 
discontinuities. Another application is to compute new slices in others planes than 
the set of initial slices. Satisfactory results are even obtained with a limited number of 1 
slices. (Fig. 2,4b) 

Major areas of application are cranial and skeletal imaging (Fig 1). 3D 
visualization helps the surgeon to perform a surgical planning: the aim is for instance 
to determine the presence and extent of congenital or acquired, craniofacial or 
orthopedic deformities. Efficient and reliable results have been obtained in clinical use. 
Another area of application is Computed Assisted Stereotaxy [LAV 89] where the 
surgeon can select the optimal trajectory to introduce a needle into the brain (Fig. 2). 

3 3D BOUNDARY DETECTION 

The previously presented method makes it possible to display a surface which is 
not modelled. Object manipulation and quantization require an analytical knowledge of 
the object surface . Thus, the first step to be achieved is segmentation. The huge size 
of data makes algorithmic complexity and storage requirements the key points of 3D 
edge detection. 

3.1 State of the art: 

The classical approach consists in computing the gradient by applying an 
operator which enhances discontinuities. Most of all these operators are 3D 
generalization of 2D edge detectors [MOR 81]: Roberts [LIU 77], Hueckel [ZUC 81], 
Canny [MON 89], Marr and Hildreth operators [HOH 88]. Critical problems of many 
of these detectors concerns the size of die convolution mask, edge points linking, gap 
filling and false edge points eliminating. 

A possible solution is to use "snakes" [KAS 87, AYA 89] but up to now their 
extension to the 3D case raises specific problems. 

Through cuberille modelling many surface tracking algorithms have been 
developped ranging from a simple thresholding to a cost function [HER 88, GOR 89, 
CAP 89]. But these technics are very sensitive to inter-slice distance. 

3.2 Methodology: 

The continuous modelling allows stable evaluation of differential operators such 
as gradient or laplacian. However, tracking the surface of an object can be proved to be 
equivalent to finding a stable invariant manifold of a system of differential equations. 
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Finding this manifold turns out to be a particular case of surface intersection problems 
[PHI 84] which provides an immediate analytical representation of the surface. 

The other major advantages of this method are to perform segmentation and 
surface tracking simultaneously, to describe complex structures in which branching 
problems can occur, and to provide accurate and reliable results. 

Let us first consider the 2D problem. We are looking for a boundary surrounding 
an object with an approximatively homogenous density f, thus verifying: 

f(x,y) = constant (Eq. 2) 

This curve is represented with parametric coordinates by : 

x = x(t);y = y(t) 

The continuous modelling involves first derivative existence of f; then a solution 
should verify the following equation 3 obtained by differentiation of (2): 



x'(t)*-^.+ y , (t)*-^ = 0 

dx 3y (Eq. 3) 

A particular solution of (3) is : 

xX0= (Eq.4) 



But this system does not provide a stable solution: a perturbation (due to noise) which 
would move the curve away from the level line could not be corrected. That is why we 
add a component to the system (4) which brings the curve back to the level curve 
"f(x,y)=constant M according to the greatest slope line of the (f-constant)2 function. We 
thus obtain: 



x'(t) = |£- p*^*(f-constant)/Gf (Eq.5) 

y,(t) = " IT" P*^*(f-constant)/Gf 

where Gf= ft* |L + |L*|L 
dx dx dy dy 

This system consists of two parts: the first one corresponds to "edge tracking" 
component and the second one is a kind of "elastic force" which allows noisy image 
processing. The p parameter allows to balance these two terms. The system (S) may 
be solved by numeric analysis methods with initial conditions. The parametric 
representation of the curve is straighforwardly obtained. Let us remark that this 
differential system may be built in any plane of the 3D volume. 

Particular points of the object surface such as local extrema can be found by 
similar methods [MAR 89]. 



3.3 3D Strategy: 

We are looking for a surface verifying f(x,y,z) = constant 
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Hie data geometry has induced our parametrization of the surface we are looking for : 

x = x(tji);y = y(t t h);z = z(h). 
Our boundary tracking method is as follows: 

The algorithm starts with a point of the surface and with a caracteristic density 
value. For each k slice, the differential system is solved in order to obtain a closed 
curve. From some points of this curve, we follow the object surface until the next 
(k+1) slice by building new 2D differential systems in slice-perpendicular planes. The 
ends of these curves belong to the (k+1) slice and are thus used as starting points to 
find one or several contours on this slice. Such a search is also performed towards 
previous slices: all object components can then be found. It is already an error 
recovery mechanism because links previously found between contours are verified in 
the same time. 

The algorithm stops when all slices have been processed or when the object 
surface has been entirely described. 

Other segmentation control process are based on global statistical or geometric 
properties. They use simple data such as density, gradient along the contour or the 
curve segment, surface, length and allow to correct some errors due to noise. These 
control processes are general enough to be applied to all cases. More specific a-priori 
knowledge might be introduced at this level to improve segmentation control for 
particular objects (convexity for instance^iumber of branches,...). 

This algorithm was tested on CT (Fig 3) and MRI images (Fig 4). 

Homogeneity is not always a stable characteristic of an anatomical structure. 
Now we are working on differential system building which performs H(f) = 0, where 
H(f) is an operator as the laplacian or Marr-Hildreth detector. A possible extension of 
our method is to include a model of the object of interest This model is described 
with spline functions and will be iteratively modified to fit with the real object 

4 SURFACE MODELLING 

4.1 State of the art: 

Segmentation step usually provides a set of contours. A surface modelling based 
on these contours is needed to allow shaded surface display. A solution is to create 
intermediate contours by dynamic elastic interpolation (LIN 88) and to display the 
surface on a Voxel Processor. Triangulation techniques have been applied to compute 
contour interpolation (FUC 77, BOI 88). But branching description sometimes 
requires user-driven interaction. 

4.2 Our Method: 

In our case, this problem is solved at the segmentation level. Our algorithm 
allows to automatically find all object components and to link them together (see § 
3.3). The information brought by the segmentation step allows to easily model the 
surface with surface patches by means of the set of contours and paths linking them 
together. Thus, this technique achieves branching description without any user-driven 
interaction. 

4.3 Applications: 

Shaded surface display, simple object manipulation (like rotation) and 
quantization use classical algorithms on surfaces defined by their parametric 
representation. 
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Now work is in progress to apply the presented continuous modelling to 
surgical planning and to matching of multi-modality information. For instance, in 
Computer Assisted Neurosurgery, the surgeon needs to match intra-operative 3D 
images (from CT or MRI) with per-operative X-ray pictures [LAV 89 & 90]. We 
suggest to use only reference 3D internal anatomical structures (the ventricular 
system) : an iterative 3D/2D matching algorithm on segmented surfaces is performed 
Thus the surgeon's strategy, chosen on 3D images, to reach the target (a tumour for 
instance) is correlated with intra-operative information. 

5 CONCLUSION 

Some objectives of a 3D medical analysis system such as visualization, 
manipulation and quantization of 3D internal structures of patients have been achieved. 
This "basic tool" is going to allow further specific developments in Computer 
Assisted Medical Interventions (CAMI) (see [MUR 90] for a review of existing 
applications and of work under development). 

Work in progress also concerns improvement of the 3D segmentation step in 
order to include an anatomical model of the object of interest We named this approach 
"spline snakes" because of snake similarity. The major advantages are as follows: the 
model is described by a limited number of control vertices and a 3D extension is 
possible easilier. 
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Fig. 1 : 3D display of the skull reconstructed from Computed Tomography images 
before (left) and 




Fig. 2 : Interactive selection of a target and a trajectory on MR images. Reconstructed 
oblique sections arc superimposed on a 3D display of the head (obtained by processing 
20 MRI slices) 




Figure 3 : Segmentation 
of a CT image of a femour 
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Fig. 4: Results of a tumour segmentation (MRJ) 

(a) edge detection of the central part of the tumour 

(b) segmentation in a plane perpendicular to the initial slices 

(c) 3D display 
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Abstract 



A prototype knowledge-based system, Image Retrieval Expert System (IRES), is 
being developed at the University of Arizona. This system is to be coupled with the 
distributed database system of Picture Archive and Communication Systems (RACS) 
to achieve high speed image retrieval system performance required by radiologists. 

IRES is an automated tool which predicts and retrieves "old" images required for 
immediate comparison with present images. It is being developed through the use of 
radiologists* expert knowledge in image retrieval The coupling of IRES with PACS 
Distributed Database (DDB) should, therefore, shorten PACS system response time. 
It should also save radiologists' time in selecting films, minimize the turnaround 
time of the exam interpretation function and improve the diagnostic effectiveness 
and quality by providing relevant and sufficient images automatically. 

In this paper, the demand for IRES to select "old" and "relevant" images from 
different anatomical portions and different modalities as well as the heuristic rules 
are described. As heuristic rules are implicit and experts are less available, special 
efforts have been made to obtain expert knowledge The development of the IRES 
prototype illustrates the new application of expert system in the radiology department 
Moreover, it provides guidelines for future system development. 



1 Introduction 



Picture Archive and Communication Systems (PACS) are undergoing rapid devel- 
opment to cope with increasingly qualitative demands about processing capabilities, 
efficiency, and the capacity of the system from the user. Solutions for satisfying 
these demands may be divided into hardware, software, and intermediate approaches. 
Hardware solutions include high-capacity fiber-optical networks, high capacity/fast 
data storage systems [1], special workstations [5], etc. Distributed database design 
is an example of an intermediate approach [3], We herein propose software ap- 
proach, an expert system for image retrieval named Image Retrieval Expert System 
(IRES). It is designed to better meet the response time requirement of image retrieval, 
considering a reasonable cost/performance tradeoff. 
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The basic idea and functions of IRES were conceived following careful obser- 
vations of radiologists' exam interpretation (reading) activity for the requirement 
analysis of PACS. Retrieval of old images is necessary when radiologists need to 
compare a newly taken exam with "old exams" of the patient undergoing diagnosis. 
Currently, film librarians retrieve the whole file of images of the patient and put 
the most recent related images taken by the same modality for the same anatomical 
portion onto the j/iewbox, or alternator, etc. Radiologists compare images on the 
viewbofces while occasionally fumbling through the remaining images in the film 
jackets for related images in a matter of a few seconds. In order for PACS to equal 
the performance for "old" image retrieval of the current manual film-based system, a 
software system needs to carry out at least two tasks achieved by film librarians and 
radiologists. First, because a single image Lakes several megabits and the amount 
of images in a hospital is large, a multilevel data storage system is proposed by 
placing "older" images in less accessible and slower storage devices to minimize the 
cost [4]. To reduce system response time in this design, a means is to be devised 
to pre-fctch relevant images to local buffer memory. Second, because not all "old" 
images are necessary in the film interpretation task and the local buffer memory 
is limited, detailed rules used by radiologists to select "relevant" images should be 
identified and automated. Considering these functions, we propose IRES should pre- 
fetch images to local buffer memory using a set of heuristics rules acquired from 
expert radiologists. 

Like any other expert system (ES), the development of IRES has to go through an 
iterative design life cycle including knowledge acquisition (KA), knowledge analy- 
sis/representation, implementation tool selection, prototyping, testing and installation 
[6]. Among these phases, KA involves two groups of people, knowledge engineers 
and experts, and determines the system usability and performance which depend on 
the accuracy of knowledge acquired from radiologists. Therefore, methodologies, 
activities, assumptions, and results derived in KA will be accounted firsL The ar- 
chitecture of the IRES prototype will be described. The advantages of IRES and its 
impact on PACS performance will be addressed. Finally, the achievements in the 
efforts for building the IRES prototype are presented and future works to be done 
arc addressed, 

2 Knowledge Acquisition 

The development of the IRES prototype started with knowledge acquisition. Knowl- 
edge engineers first collected information from the domain environment to achieve 
a general understanding of the internal process of the domain and then generated 
some assumptions. Those assumptions were verified with experts. Two methods, 
interviews and GDSS sessions, were used to acquire the correct knowledge from 
single and multiple experts respectively. This section describes important activities 
in the knowledge engineers' efforts to elicit heuristic rules and the results from that 
process. 
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2.1 Initiation of the IRES Prototype 

Through site observations, knowledge engineers learned the processes and data flow 
of the radiological exam interpretation function. In the current manual system, radi- 
ologists are scheduled to interpret radiological exams. As mentioned in the introduc- 
tion, film librarians prepare and retrieve images to be read. The retrieval activity is 
"routine" because the whole film jacket of images are retrieved for each patient who 
takes new exams. Yet, partly due to the limited space on viewboxes or other viewing 
consoles, only the most recent relevant exam is posted. Radiologists occasionally 
select images from the patient's film jacket to better achieve diagnosis task. 

Our site observations of the exam interpretation function allowed us to conceive 
several important criteria for retrieving images. They are described later along with 
the observations from which the assumed retrieval criteria are derived. They include 
1) patient name - images of the same patient are compared; 2) age - age is on the 
consultation requisition form (CRF) which is attached with the newly taken exam; 3) 
sex - sex is found on the CRF; 4) section - section is on the CRF; 5) current disease 
- suspected disease is on the CRF; 6) medical history - medical history is retrieved 
along with the old exam; 7) modality - images taken of the same modality are usually 
compared; 8)anatomical portion (AP) - images for the same AP are compared most 
frequently; 9) procedure number - procedure number has direct co-relation with AP; 
10) age of films - recent images are most frequently retrieved. 

We also observed that under different hospital scenarios/activities, the retrieval 
criteria may change. The different hospital scenarios include emergency, surgery, 
conference, inpatient diagnosis, and outpatient diagnosis. 

Additionally, wc made use of the statistical results of a survey conducted in 
March 1989. Having become aware of retrieval criteria, various scenarios for image 
retrieval, we continued our efforts to explore and verify the exact "rules" for image 
retrieval employed by radiologists by means of interviews and GDSS sessions. 

2.2 Knowledge Acquisition Methodologies 

Interviews and GDSS sessions are techniques used to verify the observed image 
retrieval criteria and explore new ones. The interview is a traditional approach for 
acquiring specialized domain knowledge from experts. It is also time-consuming. On 
the one hand, knowledge engineers have to spend time arranging appointments for 
initial or follow-up interviews, preparing/designing questions, recording discussions, 
analyzing interview results, etc. One the other hand, domain experts have to spend 
time meeting and understanding what knowledge engineers wants to know. Generally 
speaking, only one expert participates in an interview. 

Two types of questions were asked in interviews. The first type aimed to clarify 
and verify the assumptions about image retrieval criteria and scenarios derived from 
our general observations. The second type focused on cases which represented the 
specialty of the expert being interviewed because radiologists usually specialize in 
one area. 

In the interviews using the first type of questions, we asked engineers to comment 
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on the relative importance of each retrieval criteria and to describe image retrieval 
activities in each scenario. In the interviews using the second type of questions, we 
first decided cases to be studied based on the specially of available experts. We were 
concerned with cases involving multiple- modality retrieval, cross-anatomical por- 
tion retrieval, or selection of exams from a large amount of relevant images. Four 
target areas were chosen, namely pediatric cases in cardiology, surgery in cardiol- 
ogy/orthopedics, and chronic cardiology disease. The experts further narrowed the 
scope by concentrating on congenital heart defects for pediatric cases in cardiology, 
coronary heart disease for chronic cardiology disease and other surgery related cases. 

A Group Decision Support System (GDSS) at the Department of Management 
Information Systems at the U of A was utilized to verify and obtain knowledge from 
multiple experts simultaneously. GDSS provides an electronic form of communica- 
tion between knowledge engineers and experts as well as among experts. Domain 
knowledge is obtained from multiple experts and organized using GDSS facilities. 
Additionally, conflicting knowledge can be reconciled by immediate discussion at 
GDSS sessions using GDSS tools. 

It takes a great deal of preparation to effectively use GDSS sessions in knowledge 
acquisition. Knowledge engineers have to coordinate GDSS sessions, invite experts, 
contact the GDSS facilitator, prepare GDSS session agenda/questions, and arrange 
follow-up GDSS sessions or interviews, etc. As this writing, one GDSS session had 
been held with the coordination and assistance from the management of the Radiology 
department at University Medical Center (UMC). Since each physician attending the 
session had his own specialty, questions asked in this GDSS session aimed at having 
physicians brainstorm on general cases and reconcile any conflicts that arose. Due 
to this dynamic brainstorming process and the participation of multiple experts, the 
amount of knowledge acquired in one hour is much more than that acquired in a 
one-hour interview. Group meetings can spare knowledge engineers from conducting 
repetitive procedures in KA and ensure the accuracy and consistency of acquired 
knowledge immediately. 

2.3 Results from Knowledge Acquisition 

In the interviews seeking to assure the validity of assumed retrieval criteria, it was 
found that almost all of the conceived image retrieval criteria were relevant, but some 
corrections and additions needed to be made. Sex has no significant relationship with 
image retrieval. However, different disease types, such as acute and chronic, will 
result in varied retrieval patterns. It was also found that not all of the scenarios 
required image pre- fetching, one of the primary functions of IRES. The contribution 
of IRES will only be found important in those scenarios in which image retrieval 
is routine (as opposed to ad hoc in conference scenario), such as cases of surgery 
preparation, inpaticn t/outpaticn t diagnosis, etc. In the interviews concentrating on 
selected cases, heuristics rules are obtained pertaining to them. For example, in 
our study in cardiology, we found that CT was never used for cardiology cases. 
The GDSS session allowed us to obtain more accurate and varied rules due to the 
different backgrounds of participants. 
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3 IRES Prototype 

Based on the results obtained from KA sessions, an IRES prototype was developed 
to quickly obtain feedback from radiology experts, evaluate ES tool and specify 
the performance criteria for IRES. Connections between a knowledge-based system 
(KBS) and a database system (DBS) fall into three categories, system enhancement, 
KBS and DBS coupling, and integration of KBS and DBS. Among them, KBS and 
DBS coupling is perceived as a suitable architecture because both IRES and PACS 
are kept intact and independent, yet interrelated. The IRES prototype involves three 
major components, the databases, the procedural control algorithm, and the rule 
base. The declarative knowledge in the databases and procedural knowledge within 
the rule base constitute the knowledge base of IRES. The databases are managed 
by a DBMS and can be accessed by ES. The rule base consists of 44 heuristics 
rules of image retrieval presented by the production system. The procedural control 
algorithm imposes an external control over the activation of rules. Currently, EXSYS 
was chosen as the expert system tool to implement the components. We assumed 
that the connecting DBMS would be responsible for retrieving information from 
the databases according to the specifications generated by the IRES prototype. An 
interface, a text file, is used to achieve this communication. It should be noted that 
the operation of the system is invoked in batch and no interactive consultation is 
supported. 

4 Advantages of IRES 

IRES is designed to overcome the bottleneck in system performance caused by the 
large amount of data in a hospital context, as well as the large amount of memory 
space required by each image. Hence, by introducing IRES, image transmission can 
be expedited by pre- fetching relevant images at off-peak time periods and the amount 
of images to be transferred can also be reduced with selective retrieval rules. Thus, 
the response time can be greatly improved. Due to the improvement in performance, 
the exam interpretation activity involving IRES can be expedited. 

IRES also provides the valuable advantages of most expert systems. With the 
use of knowledge acquisition methodologies, knowledge engineers acquired implicit 
and ill-structured domain knowledge from experts and organized the knowledge into 
a form suitable for the use of an automated system. The organized knowledge, then, 
can be preserved regardless of the absence of experts. Given the ability to preserve 
knowledge, explain results, ES serves as a training tool for novices in the domain 
area. The advantage is important because the accumulation of past experiences and 
education is necessary to refine the state of the art techniques in this particular 
medical field. 
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5 Achievements and Future Works 

IRES is needed to improve the performance of a PACS to meet the radiologists' 
response time requirements. Although the IRES prototype is simple for the moment, 
it landmarks several significant achievements in this research. The IRES prototype 
allows us 1) to verify the assumptions of image retrieval criteria under different 
scenarios; 2) to compare and identify effective knowledge acquisition methods; 3) 
to experiment the encoding of heuristic rules; 4) to engage experts in the process 
of IRES development; 5) to estimate the future work needed and outline a detailed 
design for a complete IRES. 

One of the contribution of the IRES prototype is to obtain quick feedback from 
the user. Feedback can be obtained by displaying demos followed by interviews or 
questionnaires with individual experts, general group discussion using GDSS tools, or 
panel discussion in each section. To improve the effects of testing, better explanation 
and learning capabilities should be incorporated in the system so that the user is 
allowed to employ, understand, and comment on the system behavior. Moreover, 
the user may input new rules in the direct interaction with the system. The learning 
capabilities of the system should be so powerful that experts only need to input new 
knowledge in a form similar to that of natural language or spreadsheets/databases 
without worrying about the embedded formal knowledge representation. 

Much work needs to be done to expand the current IRES prototype. More image 
retrieval criteria are to be acquired. A GDSS session will be of more use as the 
efficiency of the process and the accuracy of the knowledge are better achieved. More 
cases, especially complicated cases such as tumor, cancer, and heart transplant, need 
to be studied. Furthermore, we propose to adopt a look-ahead two-stage database 
access scheme to reduce the frequency of image access. In this scheme, IRES 
will first issue queries to surrogate files to decide whether the image requested arc 
available. Once the response is positive, IRES issues formal queries to image files 
through an interface to DBMS. While one of the aim of IRES is to reduce the 
amount of images being retrieved with selective rules, temporal information should 
be recorded and help provide effective functions in selecting samples from a large 
number of images taken during a time span. Additionally, an interface, Workstation 
Queue Manager, is to be devised to decide at which workstation to put the retrieved 
images when multiple workstations arc envisioned in future PACS. 

Several tasks need to be done to achieve the proposed connection between KBS 
and DBS. It was also determined IRES is to be connected to a SQL-based DBMS, 
Sybase, A set of investigations on Sybase are to be conducted. A proper site for 
IRES, which is regarded as an independent system, is to be determined and other 
expert system shells which can be linked with Sybase should be studied. While 
considering the linkage with a distributed database management system, IRES should 
intelligently help avoid bottlenecks in data migration by accessing data about image 
location and decide when and where images should be efficiently retrieved [2]. 

In sum, it is believed that more research efforts arc required to make IRES 
functional and useful in PACS. 
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Abstract 

Many software and hardware systems exist to produce realistic images of 3D 
medical objects. The images they present are becoming familiar to the clinical 
community. With this familiarity comes a demand for more sophisticated facilities. 
The ability to satisfy these demands depends in part on the data representation 
employed. In particular, the ability to cut away parts of an object and look inside 
requires a volume representation, and ideally one retaining the grey level resolution 
of the original data. In many systems such cutting away is achieved by specifying 
a number of half-space clipping planes that intersect the data. Such techniques are 
limited in the volumes of interest (VOIs) that they can isolate. The requirements for 
Computer Simulated Surgery (CSS) include the isolation of arbitrarily complex 
VOIs, and the manipulation and visualisation of multiple objects simultaneously. 

The UCL3D system uses a combination of greyscale data and volume masks. A 
number of facilities are presented that operate on Boolean expressions of objects and 
masks. Thus expressions can be visualised, translated, rotated, and merged. A 
combination of these facilities allows an interactive CSS system to be developed. 
Results applied to facial surgery are shown. 

1 Introduction 

Computer Simulated Surgery (CSS) is becoming recognised as an important 
application of 3D medical imaging systems [Hemmy 1983, Vannier 1983, Cutting 
1986, Tessier 1986, Udupa 1986]. From a Computer Scientist's point of view the 
requirements may be summarised as data acquisition, preprocessing, segmentation, 
visualisation, manipulation, quantification, and registration. The first four of these 
elements have received considerable attention. Manipulation is the main feature 
discussed in this paper. 

A question of interest is the appropriate representation. Early systems performed a 
static segmentation (thresholding, surface tracking, region growing etc.) to produce 
a binary object that could then be rendered (displayed) on a 2D computer screen. 
More recently dynamic segmentation techniques have developed that perform a 
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decision per voxel during the display algorithm, or else produce a "fuzzy" display 
by volumetric integration [Heffernan 1985, Levoy 1988, Trousset 1988]. 

When objects are to be "dissected" and manipulated, such fuzzy techniques would 
appear to be inappropriate, since an object needs to be unambiguously defined 
before it can be transformed. However, the approach outlined here is to dissect 
objects by defining a volume mask, or Volume of Interest (VOI) that is simply a 
binary partition of the data space. Subsequent operations are carried out on an 
expression (e.g. a Boolean expression) of the mask and the original data set. This 
allows a flexible surgical simulation environment to be developed. 

Figure 1 shows an overview of the system that has been implemented. The rest of 
this paper will outline the means of mask generation and expression evaluation, with 
some examples from maxiUo-facial surgical planning. 



2 Volume Mask Generation 

In the Surgical Simulation application, the user most often wishes to view a dataset 
and interactively demarcate sections to be modified. Thus it is useful to draw the 
required dissection and observe it in multiple views, editing as required. The process 
starts with the object Q and displays it in a number of views to obtain a set of 
displayed images {Dyj}. For each view a number of arbitrarily complex regions {RJ 
are drawn on these images, where each Rj may have separately connected areas. 
Then the inverse projections are derived to form a cylindrical volume (CV) and an 
iterative method of forming the VOI is allowed by considering at any stage the 
partial intersection set : 

(VOI) k = CV X n CV 2 n ... n CV k (1) 

and creating each subsequent display as one of 

D yk + i = Display(Q) under transform T M (2a) 
Dy k *i = DispIayJBoolean(Q n VOI0 under transform T k+1 (2b) 
Dy k+ i = Display_Boolean(Q - VOIJ under transform T k+1 (2c) 

Thus some flexibility is allowed in creating the VOI. The basic philosophy is to 
use as many images obtained by equation (2) as are useful to provide the visual 
cues for creating the VOI. 

The actual CVi's are intermediate and of no interest. Instead the VOI is built by 
editing the volume described by equation (1) at each stage. 



3 Expression Evaulation 



In the current system the data and masks are both stored in a hierarchical octree 
representation, the former as grey scale and the latter as binary. The purpose of this 
mask is to allow operations on the original data, inside or outside the volume 
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ACQUISITION 



PREPROCESSING 




represented by this mask. Thus algorithms are required that operate on Boolean 
expressions of octrees. Examples are : 



Boolean (n, u, -) 

DisplayBoolean (n, u, -) 
Translate_Boolean(n, -) 
Rotate_Boolean (n, u, -) 
Segment BooIean (n, u, -) 
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These are algorithms operating on pairs of trees, related by one of the Boolean 
operators Intersect, Union, Difference. Usually one of the operand trees is a binary 
VOI, and the other either a binary or grey data tree (Arridge 1989). 

Explicit Boolean operations are required actually to create merged and split objects. 
The implementation of Boolean algorithms amounts to the evaluation of a simple 
rule set at each node. The rules for the three simple Boolean operations shown in 
Figure 2 where P/E is written because the intersection or difference of two Partial 
nodes might give an Empty node, (but not a Full node), and P/F indicates that the 
Union of two Partial nodes might give a Full node (but not an Empty one). 



4 Examples 

Patient 1, aged IS, was diagnosed as Crouzon's Syndrome (craniofacial dysotosis). 
The surgical procedure adopted is the Kufner osteotomy. It is a cut taking in the 
lower orbits of the eyes, to separate the maxilla. This whole piece is then advanced 
forward along the occlusal plane to correct the open bite. For this patient the 




INTERSECTION DIFFERENCE UNION 

Figure 2 : Boolean operations on octree nodes. 
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advancement was estimated as 1.5cm. 

The patient was scanned on a GE9800 with 35 slices. The voxel size was 0.86 by 
0.86 by 5mm. It was necessary to remove the headrest from the scans by 2D ROI 
editing. Some artefacts were also removed. 

The simulation stages were as follows : 

A - Display(Patient) in Anterior, Left, and Right views. 

B - Marie region R t in the Anterior view and CreateMask VI = VOIj 

C - Display_Boolean(Patient n VOIi) in Left view. ~ 

D - Draw region on this view, and Edit_Mask(Vl). 

E - Specify the vector of translation by marking the anterior of the maxilla at 

one end, and the final position so as to cause closure of the open bite (T 3 ). 
F - Form a new object (Translated_Maxilla) = Translate_Boolean(Patient n VI) 
G - Form the final object Prediction = Boolean(Translated_Maxilla u (Patient - 

VI)) 

Figure 3 shows stages A to C of the above. The Anterior, Left and Right views 
are displayed with ROI, drawn on the Anterior view. The expression (Patient n 
VOI,) is shown in the bottom right, with ROI 2 drawn on this view. The translation 
vector was drawn by marking arbitrary points in space to close the open bite. The 
movement specified was reported as : (Forward 2mm Downward 1mm). Figure 4 
shows the resultant of stage G in Left and Anterior views, contrasted with the same 
views of the original patient. 



5 Conclusions 

The need for a sophisticated surgical planning package includes the ability to 
dissect, transform and merge pieces of object In the UCL3D system this is achieved 
using hierarchical volume descriptions of objects and masks, and standard octree 
algorithms for the manipulation. This volumetric approach combines the advantage 
of complete access to the data, with a space planning ability similar to CAD/CAM 
applications. 
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Figure 4 : The original and predicted hard tissues. 
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Validation and Preliminary Clinical Trial of a 
Radiology Expert Computer System 

Charles E. Kahn, Jr.fr® 
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* L 
Abstract The PHOENIX Radiology ./Consultant is a rule-based 

expert computer system which assists physicians in planning radiologic 

work-up strategies. This article describes the process by which the 

system's knowledge base was created and validated. Also, to test the 

feasibility and acceptability of the system, PHOENIX was placed in 

clinical trial for more than one year. During a period of 58 weeks, the 

system was used 473 times, an average of 8.2 times per week, primarily 

by medical students and nonradiology physicians. Much of the system's 

use occurred at night and on weekends, when the radiology department 

was not fully staffed. These findings suggest that an expert system can 

serve as a useful and informative component of a radiology information 

system. 



Several investigators have applied artificial-intelligence 
techniques to problems in radiology such as procedure selection and 
differential diagnosis [1-3]. As with many other applications of 
artificial intelligence in medicine, these expert systems are 
developmental. To this author's knowledge, these systems have not 
been placed into clinical use and have not been evaluated thoroughly 
and systematically. This article discusses the validation and the 
preliminary evaluation of a radiology expert system. 

The PHOENIX Radiology Consultant [4] provides information 
about the proper selection of radiologic imaging procedures. The system 
primarily was designed to aid nonradiologists, in particular, the 
housestaff physicians and medical students at our university teaching 
hospital. For a specified clinical problem, PHOENIX offers a recom- 
mended work-up strategy in the form of an algorithm or "decision tree"; 
physicians can view the algorithms and "move" through the decision 
tree to obtain more detailed information about the relevant rules and 
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imaging procedures. PHOENIX has knowledge of 54 common clinical 
problems, such as head trauma and pulmonary embolism, and of more 
than 75 radiologic procedures. 

The algorithms encoded into PHOENIX's knowledge base were 
derived from our radiology department's imaging protocols as well as 
from several widely accepted references [5-7]. The algorithms were 
modified where necessary to ensure compliance with departmental 
standards. Three radiologists encoded the algorithms into production 
("if-then") rules and verified the accuracy of the knowledge base by 
comparing the computer-generated algorithms with the source 
documents. Before full release, PHOENIX was made available only to 
our radiology department's section chiefs, who reviewed the computer- 
generated algorithms. This step provided a "peer-review" process to 
assure the accuracy of the computer system's knowledge. If the imaging 
work-up suggested by the computer system differed significantly from 
their own work-up strategies, these physicians alerted the system's 
developers. After several minor technical changes were made to the 
algorithms, the imaging strategies were considered valid and suitable 
for general release. 

The system was made available to members of the radiology 
department in October 1987. This limited release enabled all attending 
and resident radiologists to evaluate the system and to detect any 
discrepancies between imaging strategies recommended by the computer 
system and those actually recommended by the department's 
physicians. PHOENIX was released for general use in December 1987. 
The system is available to members of our radiology department - staff 
radiologists, residents, technologists, and appointment clerks - at 
terminals distributed throughout the radiology department. 
Nonradiology physicians and medical students can gain access to the 
system from one terminal in the main radiology viewing room. 

From June 15, 1988 until July 26, 1989, a period of 58 weeks, all 
encounters with the system were recorded. For each clinical problem 
selected by a user, the computer recorded the date and time, the user's 
ID code, and the location of the user's terminal. Medical students and 
nonradiologist physicians could gain access to the system at one 
terminal without a radiology computer ID code. During the period 
studied, the system was used 473 times, an average of 8.2 times per 
week. By far, the largest group of users were nonradiologist physicians 
and medical students, who accounted for almost three-quarters of total 
system use. Four staff radiologists used PHOENIX during the study 
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period (16 encounters; mean 4.0 per person; range 2 - 9). Six of the 
department's 20 radiology residents used the system a total of 39 times 
(mean 6.5 per person; range 1 - 21). Seventeen radiology technologists 
and appointment clerks used the system 69 times (mean 4.1 per person; 
range 1 - 13). 

Use of the system over weekends (Saturdays and Sundays, 
holidays not included) accounted for 30% of all encounters. Almost one- 
half of the use occurred between 6 p.m. and 8 a.m., when the radiology 
department was not fully staffed and attending radiologists were not 
present in the hospital. Of 54 clinical problems, the ten most frequently 
selected accounted for almost one-half of all encounters. Disorders of 
the chest were most commonly selected: widening of the 
cardiomediastinal silhouette, solitary pulmonary nodule and 
pulmonary embolism were the clinical problems most often viewed. All 
of the clinical problems were explored at least twice. 
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Abstract 

This paper discusses the tasks that an 
Intelligent Diagnostic Radiology Workstation 
(IDRW) should be able to perform in addition 
to serving as an image review station. This 
discussion is focused on software* Hardware 
considerations are beyond the scope of this 
discussion. Included in our discussion are 
the types of data that an IDRW must handle. 
It is hoped that this discussion will lead to 
improved, friendlier, and more intelligent 
workstations which will be accepted by 
radiologists . 

This paper discusses specifications for an 
intelligent diagnostic radiology workstation. 
The term "intelligent" is used to imply that 
more than just image management, transmission, 
and storage will be performed at this 
workstation. Artificial Intelligence (AI) 
software programs will be utilized in this 
system. It will be integrated into the hospital 
and radiology information systems so that the 
system can help monitor, control, and advise 
radiographers about the performance of 
individual examinations and will assist 
radiologists in the interpretation of 
examinations. The IDRW will also serve as a 
platform for obtaining the radiology report. 
There have been several recent papers concerning 
designs for Medical Image Workstations (1-5). 
The focus of this paper is integration of 
Hospital Information Systems, Radiology 
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Information Systems, and the IDRW and the role 
of AI software in making the IDRW a labor saving 
device which can assist radiologists in their 
tasks. This paper does not discuss hardware and 
display requirements • 

One of the major problems limiting the 
integration of various computer systems is the 
lack of standardization. This hampers the 
development of IDRWs since these systems will 
have to interface with a hospital information 
system (HIS) and radiographic information system 



This paper is divided into three major portions. 
The first discusses input and output of an IDRW. 
The second discusses the functions that an IDRW 
should be able to perform. The third section 
discusses potential benefits of a IDRW. 

1 . Input and Output of an IRDW 

An IDRW must deal with three different types 
of data. These are: 



la. Patient Data 

This category of data is the data which 
is known before the radiographic 
examination is performed, i.e., patient 
history, physical examination, 
provisional diagnosis, results of 
previous radiographic examinations , 
results of non-radiological tests, and 
the reason the study was requested. 

If these data were stored in a machine- 
readable format in a hospital 
information system (HIS), then an IDRW 
could search the database of the HIS to 
identify relevant input patient data and 
present this to the radiologist. 
However, many HIS systems do not have 



(RIS) . 



a) 
b) 
c) 



Patient data, 
Image data, and 
Radiology report data. 



( input ) 
( input ) 
( output ) 
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data stored in a machine-readable 
fashion. This limits the possibilities 
for automation of data gathering by an 
IDRW. In those HIS systems that have 
machine readable databases, the lack of 
standards makes it difficult to develop 
an IDRW that can interface with 
different HIS systems, which is a 
necessity for economic viability* 

The value of having patient specific 
data from the HIS available has been 
discussed in several papers. A paper 
from Grady Memorial Hospital discussed 
the THERESA/RadARS Systems (6). This 
system maintains individual patient 
records on-line and allows interactive 
analysis of the records with an English- 
like query language. The group from the 
Latter-Day Saints Hospital described a 
system for providing a radiologist with 
clinical data such as history, physical 
exam, and laboratory results (7). This 
system is a subcomponent of the HELP HIS 
system which was developed at the same 
institution. 

lb. Image Data 

Image data includes the current image 
and previous images. Recent 
developments in picture archiving and 
communication systems (PACS) show that 
technology for transmitting radiographic 
images is becoming useful and reliable. 
Decreasing cost /time /performance ratios 
and improving the user interface may 
lead to the acceptance of these systems 
in the not too distant future. 

lc. Radiology Report 

The radiology report is the output of 
the radiologist and IDRW, when film 
review is complete. This consists of 
the radiographic features present or 
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absent on the study as well as their 
locations, the interpretation of their 
implications, and a diagnosis. This 
could be represented in a computer 
database as the presence, absence, and 
location of radiographic features and a 
diagnosis. Annotation of the digital 
radiograph may be an innovative approach 
to the radiology report. The machine- 
readable internal representation can be 
available in English for humans to 
examine . 

A variety of approaches to coding 
radiographic reports using computers 
have been tried. These have been 
reviewed in (8). These all utilize a 
common approach. First, specific 
radiographic features for each 
examination are determined. The 
radiologist then uses an appropriate 
navigation tool and marking device 
(either a computerized or a mark sense 
form) to indicate which findings are 
actually present. Some degree of 
quantification and confidence values are 
sometimes available. Recently, several 
computerized voice-to-text systems have 
been introduced to Radiology (9). These 
vary in their features: Some can 
interpret continuous speech, and some 
need interrupted speech. Some function 
using code words and features. Both the 
voice-to-text and the other systems have 
the radiographic feature as the common 
denominator of reporting syntax. 

Recently, Haug , et.al. , described an 
interesting computer program that uses 
as input free text radiologic reports 
and gives as output a list of 
radiographic findings, their location, 
and a diagnosis. This paper illustrates 
the bi-directional, one-to-one 
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correspondence between a database of 
radiographic features and a free text 
English language version of the report 
(8) . 

2. Functionality Model 

The functionality of an IDRW could be 
modeled on the patient care duties of a 
radiology resident. This model is based 
upon practice in an academic radiology 
department. In private practice, other 
allied health practioners may perform 
some of these tasks. The ability of a 
computer to perform these tasks is 
discussed. This has been divided into 
three major sections: a) before an 
attending physician arrives, b) while an 
attending physicians is discussing the 
case (with the resident), and c) after 
an attending radiologist leaves. 

2a. Before the attending arrives: 

2al. The radiology resident should be able to 
interact with the referring physician 
from the time a test is first beyig 
considered until after the report is 
complete. The resident should be able 
to discuss a patient's case and 
recommend an appropriate work-up. It 
has been demonstrated that computerized 
technology has this capacity (10-12). 

2a2. The history for each case should be 

obtained, and if necessary, a physical 
examination should be performed. While 
it is not anticipated that computers 
will be able to perform a physical exam, 
if the patient chart is available in a 
machine readable form, the history and 
physical exam data can be searched. A 
history can be taken from a patient by a 
computer . 
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2a3. The type and format of certain studies 
must be specified. This could be 
performed by an advice system based on 
the information gathered above (13). 

2a4 . The radiology resident may do a study 

such as a barium enema. This would not 
be feasible with anticipated technology. 

2a5 . The current films should be loaded on 
the alternator. This is done by PACS 
systems . 

2a6 . Appropriate comparison films should be 
pre-fetched and loaded. This is not 
done by PACS systems currently; however, 
solutions have been proposed elsewhere 
(14). Radiologists may have different 
preferences for the layout of the images 
of the current and comparison studies on 
the IDRW. For example, on a 4 screen 
display one may prefer the current PA 
and Lateral chest x-ray on the middle 
two displays, with the respective 
comparison images on the outside 
displays, but another may prefer the 
current study on the right and the 
previous study on the left. The format 
for display could be individually 
designated in a personal preferences 
file' in the IDRW. 

2a7 . Each case should be examined as soon as 
performed to determine if the study is 
complete and to form a preliminary 
diagnostic impression of the study. 
This is not possible with computerized 
systems . 

2a8. The resident should have knowledge of 
pathology, radiographic features, and 
image artifacts. Artificial 
Intelligence technology may allow an 
IDRW to "know" about pathology. This 
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knowledge is essential because it must 
be determined if the examination has 
adequately demonstrated the pathology 
that may be present before it can be 
considered complete. Some computer 
programs are capable of identifying 
specific radiographic features (15-17). 

2a9 . The size of lesions should be measured 
and recorded. This could be performed 
with machine assistance (18). 

2alO.The attending physician should be 
informed of problem cases to allow 
timely intervention. This could be done 
by a machine based on information 
gathered in 2al and 2a2 . 

2all.The attending physician should be 

informed when enough cases have been 
completed and it is time to review 
cases. An IDRW can indicate when enough 
cases have been assembled to allow 
review. 

2b. When the attending arrives: 

When the attending physician arrives to 
review, the history, the radiographic 
findings, the implication of these 
findings, and possible diagnoses for 
each case, should be discussed. If an 
IDRW has performed some of the data 
collection and image analysis functions 
discussed in 2a, then this can be 
presented to the radiologist. The 
discussion with the attending physician 
should be adequately documented. This 
is done by writing a brief statement in 
a log book. 

2c. After case review: 

2cl. After the case review session, the cases 
should be dictated based on the notes 
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taken during film review. This can be 
performed interactively with an IDRW if 
the appropriate software and interface 
are developed. 

2c2. Films should be removed and directed to 
storage, and any other paperwork 
involved should be cleaned up. 

2c3. Referring physicians should be contacted 
about significant radiographic findings. 
If a phone number or beeper code could 
be autodialed by the workstation, it 
would ease this task. Machine to 
machine direct transmission of this 
information is also possible. An 
interface has recently been announced 
for the Kurzweil AI 1 s VoiceMED system 
which allows the faxing of reports to 
physicians offices upon completion (19). 

3. Discussion 

In this section, I will discuss the benefits 
of having an intelligent diagnostic 
workstation which integrates patient data, 
image data, and report data. This will be 
divided into discussions of quality control 
and research; however, there is overlap of 
these two categories. 

3a. There are several quality control 

features that can be implemented on an 
IDRW as discussed above. These consist 
of consistency checking, advice systems, 
order entry advice, and automated 
follow-up. 

Consistency checking can be implemented 
on an IDRW. If a descriptor has changed 
between registration and report 
generation, i.e., the left knee is now 
called the right knee, the computer 
could test for this condition and point 
it out. 
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Embedded advice systems can examine the 
database of radiographic features the 
radiologist has observed for a case and 
offer independent advice as to a 
possible final diagnosis. This can be 
done with either advice systems or 
critiquing systems (20). This may 
prompt the radiologist to reconsider the 
diagnosis . 

Computerized order entry with critiquing 
of choices can advise referring 
physicians of appropriate work-ups. 
This can act as a screen for 
inappropriate studies . 

Automated follow-up of cases may be 
possible to allow the patient's final 
discharge diagnosis to be correlated 
with the radiographic reports that were 
generated during the patient's 
admission . 

3.b The research benefit of an IDRW stems 

from integration of the Radiology report 
and patient specific information for 
research on patient outcome. For 
instance, for those patients who have 
ventilation/perf usion lung scans one 
would like to know if and when they have 
angiograms. One should be able to ask 
the IDRW to inform when this occurs. In 
addition, it should be easy to obtain 
cohorts of patients who have had a 
particular type of study or diagnosis 
for research and quality assurance 
purposes . 

4. Conclusion 

Incorporating some of these tasks into an 
IDRW should increase throughput, accuracy, 
income per unit time. This will lead to the 
acceptance of IDRW into radiology departments 
as labor saving devices. 
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ABSTRACT 

In order to fully understand the network and archive require- 
ments needed for PACS of the future, we believe it is important to 
move beyond models which are based on average daily activity. The 
introduction of actual digital system elements provides an opportunity 
to acquire, analyze and track the resulting workloads. Estimates of 
current image movement and storage requirements can provide a basis 
for sizing communications bandwidth and archive design. Alongside 
the need for data to support PACS design exists an administrative 
need for a flexible window to augment our Radiology Information 
System (RIS) in tracking departmental activity. 

We have developed a prototype Workload Characterization 
System (WCS) to meet these two needs. The design of the WCS can 
be broken into three components: acquisition, storage, and retrieval. 
The acquisition component of the design centers around the concept of 
information "probes" that collect data on differing activities around 
the department, such as patient and exam information, digital image 
generation and movement, and network traffic. The storage compo- 
nent consists of a relational database (VAX Rdb/VMS) which inte- 
grates data from information probes. The retrieval mechanism pro- 
vides an overall view of the data and the relationships among the data 
over an extended period of time. 

1. INTRODUCTION 

A Picture Archiving and Communications System (PACS) 
can be described by a deceptively simple model: 1) image sources, 2) 
image archive, 3) inquiry and display stations interconnected by 4) a 
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communications network. Many of the current PACS design efforts 
have utilized estimates of image traffic and storage requirements de- 
rived from observations of film-based operations. A phased introduc- 
tion of PACS components offers an opportunity to gather and analyze 
workload data associated with generation, storage and utilization of 
images. Dynamic tracking of these activities is needed to support per- 
formance evaluation, design assessment and PACS system 
management. 

In addition to PACS development and deployment needs, a 
timely view of departmental activity has proven essential to those 
making administrative decisions. A locally developed Radiology In- 
formation System (RIS) currently fulfills many of our existing admin- 
istrative needs. Data retrieval and presentation is based on pre- 
defined queries, and new views of the collected information require 
software development. A more flexible access, easily tailored by the 
user could enhance administrative utilization of the existing informa- 
tion. 

A Workload Characterization System (WCS) has been de- 
signed to meet these needs. Information "probes" collect data de- 
scribing department activity, and the information is forwarded and 
stored in a database as shown in the block diagram, Figure 1. The 
viewing mechanism provides an overall view of the data and the rela- 
tionships among the data over an extended period of time. The stor- 
age component utilizes the relational database (VAX Rdb/VMS) to 
support viewing the data from many different vantage points. 

2. PROBE DESIGN 

In order to achieve efficient data acquisition, the Workload 
Characterization System utilizes independent probes that provide data 
to the database. A probe can be thought of as a process that reads 
samples of data from a source and produces samples for insertion into 
the storage system. Probe data collection methods vary with the data 
to be gathered The probe can provide rate adaptation by buffering 
samples for block transfer to the database or data reduction by filter- 
ing or averaging data. 
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Figure 1. Block diagram of Workload Characterization 
System "probing" the PACS Testbed. 



2.1 RIS Probes 

Our RIS contains infoimation about film movement, patient 
information, and exam information, but the custom nature of the data- 
base inhibits the use of queries by anyone except the designers of the 
database. Patient and exam infoimation is stored in a Short Term File 
(STF) for 60 days, after which the infoimation is transferred to a Long 
Term File (LTF) where it is kept for 5 years. In addition, all transac- 
tions that update the database are kept in a running log file which is 
compared to the day's actual database updates to insure the integrity 
of the database. Two RIS probes provide data to the WCS; one moni- 
tors the STF and the other monitors the transaction log. 

The STF probe scans the RIS STF once per day, collects 
new entries and then updates the database once per day. The second 
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probe is the RIS Transaction probe, which monitors all of the transac- 
tions between the data terminals and the RIS database. Some typical 
transactions of interest are the adding of a patient or an exam to the 
RIS database or the movement of patients or film jackets from place 
to place within the department. Currendy, a prototype Transaction 
probe is designed to sample the transaction log file once per day, with 
an output sampling rate of once per day. A second generation probe 
will be able to sample the transactions as they occur, producing an 
output sample for each transaction. 

2.2 Image Probe 

Another source of information is contained in the digital im- 
ages, themselves. Currently, for example, Magnetic Resonance im- 
ages from one of our MR scanners are being transferred to a short 
term image file on the DEC 8530 cluster located within our PACS 
testbed. The MR probe executes daily to obtain information from 
these images such as exam requisition number, number of images per 
exam and number of studies per exam and updates the MR relation in 
the WCS database. 

2.3 Network Probe 

A third activity of interest is the network traffic generated 
by image sources, archives, and viewstations. An Ethernet probe pro- 
vides a statistical view of the Ethernet traffic between a group of 
PACS machines. A probe is dispatched for each image source and 
each image sink to be studied. An example of an image source is an 
MR scanner, while an example of an image sink is a viewstation. 
Typically, an archive requires two probes, one for the incoming im- 
ages and one for the outgoing images. Each probe samples data from a 
locally developed Ethernet monitoring program, "WATCH", every 
hour over a 24 hour period and packages the data for later retrieval by 
a polling program that resides on the WCS database machine. 

3. DATABASE DESIGN 

The implementation of the database is based on a relational 
database (VAX Rdb/VMS). in a relational database, data resides in ta- 
bles known as relations. In our initial implementation, a patient rela- 
tion, an exam relation, and an image relation are linked via patient ID 
and exam identification numbers. System resource activity, such as 
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CPU, disk and network utilization are linked to the patient relations 
via time stamps. Example relations are shown in Figure 2. 
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Figure 2. Example Relations in WCS 

The patient relation contains general information about each 
patient using fields such as name, social security number, number of 
visits, MIR number, etc. extracted from the primary records of the RIS 
Short Term File (and the RIS Long Term File). The exam relation 
contains information regarding actions that have taken place during 
each specific patient examination. Fields in the exam relation include 
type of exam, exam ID number, patient ID number, patient arrival and 
departure times as well as other times such as report typed, report 
signed, films done, etc. The image relation contains a subset of the 
information found in the header of the image files. Fields in this rela- 
tion include examination ID number, number of images per exam and 
number of studies per exam. The PACS resource relation contains in- 
formation pertaining to the packet flow on the LAN, the percent of 
CPU utilization and disk channel utilization. 

We can make use of data from multiple relations by creating 
logical views, consisting of a subset of records and fields from one or 
more relations in the database. To the user, a view looks exactly like a 
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relation, even though the records in the view are not stored together 
physically. We can combine information from the STF containing pa- 
tient arrival and departure times with timing information extracted di- 
recdy from the image sets to produce a more detailed look at the key 
events which occur during an examination. 

4. RETRIEVAL 

Data is retrieved for viewing using 20/20 (Access Technol- 
ogy, Inc.), a spreadsheet interface which accesses the RDB database 
through menu-driven queries. Menus prompt the user to specify crite- 
ria for the data search, data grouping, etc. The results of these queries 
are put into a spreadsheet which the user can manipulate using 
spreadsheet techniques and/or display in graph form. This interface is 
also adaptable for experts to write their own queries using the Data- 
trieve query language. Thus 20/20 makes data available to both expert 
and novice alike, creating a flexible and useful data retrieval mecha- 
nism. 

Examples to illustrate a progressively finer viewing grain are 
provided as a collage in Figure 3. The total number of exams per day 
and the number of MR exams per day are plotted for a period of one 
month. The quantity of images per MR study and the Ethernet traffic 
in bytes per hour corresponding to transmissions from one of the MR 
instruments to the image archive are shown for a twenty four hour pe- 
riod. 

5. CONCLUSIONS 

Preliminary experience with the Workload Characterization 
System illustrates the utility of the design. A collection of specialized 
probes coupled to a high-level user interface promises a sustainable 
approach to system monitoring and analysis. 
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Figure 3. Collage of example results 
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• INTRODUCTION 

This paper will give a brief history of the work that has taken place at the 
Massachusetts General Hospital's Department of Radiology since 1984 in the area 
of PACS. Additionally, it will outline the development and direction that the 
department is now taking. 

Founded in 1811, the Massachusetts General Hospital is a leading patient care, 
teaching, and research center. The 1084-bed facility, staffed with over 9,000 
employees, is the original and largest hospital affiliated with the Harvard Medical 
School. The Radiology Department employs over 70 staff radiologists, as well as 
SO plus film librarians to file and transport films. In addition, there are 51 research 
fellows as well as 34 medical residents. The RSTAR PACS development effort 
has grown over the past 5 years to the point that in January of this year it occupied 
an off site development facility in Cambridge in addition to space available at the 
main campus. 

The 340,000 annual radiology exams would generate more than 5 Terabytes of 
digital data. Because of space limitations the department maintains a 15,000 square 
foot off-campus film library in Somerville in addition to the two on-campus 
libraries. 



•PLANNING AND CONCEPTUALIZATION 

Serious discussions and exploration of how to realize Picture Archiving and 
Communications System (PACS) began late in 1984. Initially, a significant 
obstacle to defining PACS functionality was the inability of radiologists and 
others to conceptualize and comment upon systems which did not exist and which 
had no adequate model from which to extrapolate. To overcome this problem with 
product definition' a traditional industrial style marketing study was initiated 
within the radiology department. In particular, a heavily illustrated, sixty page 
user's manual for a hypothetical system was written at MGH and completed 
February 1985. The manual was distributed to all members of the department and 
shortly thereafter comments were solicited and examined. This concrete exposition 
of a PACS facilitated visualization by radiologists not previously familiar with 
PACS. The manual's distribution and subsequent follow-up proved to be a 
powerful vehicle to obtain substantial feedback from the radiology staff and others. 
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•DEMONSTRATION UNIT 

After incorporating the changes and insights garnered from the marketing study 
into our vision of a PACS, the next step in 1985 was the design and creation of a 
demonstration unit that in a very limited manner made real many of the features 
described in the revised users manual. The demonstration unit was capable of 
quickly displaying both CT, MRI, and nuclear medicine images upon multiple 
512x512 pixel by 12 bit displays. The demonstration unit, based upon IBM PC 
type machines, included pull down menus, dialogue boxes, as well as graphic tools 
for operations such as window & center. The demonstration unit responded to 
mouse control as well as conventional keyboard command. Viewing and using 
this demonstration workstation facilitated and provoked further constructive 
dialogue with and among radiologists as well as the growing PACS development 
team. 



• ixRSTAR 

From these initial 'marketing studies' the demonstration unit and users manual 
goals were established. A development program was initiated to create a first 
generation PACS at MGH. One of the first major decisions the group faced was to 
either put all our efforts into creating a (then) high resolution IK x lk stand-alone 
workstation only or a complete network based system using less ambitious 
displays. We elected to build the system as opposed to just the workstation so that 
we could gain insight and experience in issues such as image transmission using 
local area networks (LANS), statistical image caching, large multiuser 
transactional databases, multi-modilaty on-line image collection, as well as 
managing off-line and removable image storage elements. 

Knowing that this was to be at best a research system, hardware inexpensive 
enough to be discardable in the future was used. Software was to be done in a 
modular fashion to permit the forward transfer of major portions of the software to 
future versions of the system. 

The project at the Massachusetts General Hospital (MGH) became known as 
RSTAR, Radiology Storage, Transfer, Analysis and Reporting System. The first 
generation RSTAR system is known as MicroRSTAR (jjRSTAR). 

The initial uRSTAR system used a VAX 11-750 computer (Digital Equipment 
Corporation; Maynard, MA) with 1.3 gigabytes (GB) of disk storage as the 
workstation server. The medical workstations are based on IBM AT class machines 
with a MGH modified display controller. Each workstation has a graphic control 
monitor and one to four 512x512x12 bit monitors with 4 bit color graphic 
overlays. A two-button mouse is used, and the cursor moves easily across all 
screens. 

During the early development CT and MR images were entered into the system via 
magnetic tape. PET images were acquired directly though Decnet running over 
Ethernet Nuclear Medicine images were acquired through an in-house designed and 
built acquisition node which includes a floppy disk drive capable of reading 
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Technicare VIP floppies and an Ethernet interface used to convey the data to the 
VAX. 

Transmission of images to |±RSTAR workstations is done over standard Ethernet 
cable but uses an RSTAR engineered Ethernet protocol (registration numbers 
8160 and 8161). This protocol, customized for transmission of large data sets 
(images) uses die techniques of multiple packet transmission per acknowledge 
cycle and auto-routing of packet data, operates without interference with standard 
DECnet, LAT, and TCP/IP protocols on the hospital's extensive Ethernet plant. 
Image transmission rates proved to be 10 times greater than that of DECnet 
PcDos. 

MRSTAR: Software and Functionality: 

Whenever possible the RSTAR development staff purchased software, for example 
the graphic interface is based upon Digital Research Inc. GEM package. 
Nevertheless, the |iRSTAR software system included over 350,000 lines of code 
written by the MGH development group. 

A few of the operational features of the system include password access, individual 
user configuration and preference files, algorithms for the automatic presentation of 
unread cases sequentially to radiologists according to their work assignments (for 
example, CT or MR, or all cases or only undictated cases), algorithms for the 
automatic initial placement of study images on available display screens with 
consideration of the study type, available screens, and user preference. 

The initial window & center setting is accomplished automatically by an RSTAR 
histogram-based algorithm that sets four standard default buttons (for example, for 
CT they are Overall, Bone, Softl, and Soft2). Additionally, window & Center 
may be adjusted by the user via either graphic sliders located on the control screen 
or via mouse movements — vertical for window and horizontal for center. 

The workstations support a wide variety of screen display options from one image 
per screen up to 64 per screen. Screens may be arbitrarily tiled for image display 
in both number of images per screen — 1 to 64, as well as format — square tiles 
as well as rectangular tiles may be used freely upon the same screen. The 
workstation supports multi-modality viewing upon the same screen. Cine loops 
can be constructed and mixed with static images. Additionally, cine speed loop 
speed and pause are controllable. The display programs correctly handle different 
image sizes and shapes. One, several or all of displayed images can easily be 
manipulated under mouse control, including zoom and roam, flip horizontally or 
vertically, rotate 90 degrees either direction, change from gray scale to pseudocolor 
display, clear graphics/annotations, and restore image to original state. 

A wide array of measurement tools are available to the user such as length, area, 
arbitrary region of interest, and angle. All measurement results can be placed into 
VAX resident databases (Ingress from Relational Technology) or the VAX 
resident statistical analysis package (SAS from the SAS Institute). Also offered at 
the workstations but performed by the central VAX computer, are a wide array of 
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image processing tools, e.g., FFT tools, pixel normalization routines, image 
remapping routines, and image warping to name a few. 

To aid the users of the system on-line help text is available by general topic, via 
table of contents and for any menu item or icon. Electronic mail is implemented 
(useful for feedback). To allow the (iRSTAR system managers to further be of 
assistance, the workstations can be enabled to send electronic mail to the managers 
upon various user activity (for example, login). 

There are presently six (iRSTAR workstations in the department. Each of these 
workstations can retrieve and operate upon any image in the central VAX archive. 
Display time for a 512x512x12 bit CT image retrieved from the central archive is 
2.8 seconds from time of request All of the workstations have been used for the 
general display and the image processing and manipulation of CT, MRI and 
Nuclear Medicine studies. (iRSTAR was used to display vascular structures on 
MRI images very early as temporal cine data sets, prior to the widespread 
availability of commercial products. A particularly valuable, automated approach 
to Quantitative Bone Density determinations is in daily clinical use; it includes 
automated region of interest generation and calculation of data and control images 
for checking by radiologists. Nuclear Medicine digital images of all varieties have 
been archived since August, 1988; the nuclear images are available for recall and 
display for comparison to current studies. 

• Q-RSTAR 

Q-RSTAR is the name of the second generation MGH PACS system currently 
under development. With Q-RSTAR the emphasis has shifted from research to 
achieving clinical utility while continuing to test and learn from experience. An 
important Q-RSTAR goal is high resolution and high speed performance at an 
affordable price. Selected initial clinical applications will test the utility of this 
newer technology. Additionally, the goal of learning from a complete system 
continues as with the pRSTAR project 

The Q-RSTAR display will include 2Kx2.5K pixel monochrome high frame rate 
display(s) with vastly faster communications via a dual local area network. More 
technical information is provided in other papers. Q-RSTAR features 
multimodality (CT, MRI, Nuclear Medicine, high resolution digitized images, etc.) 
and multimedia capability (orchestrated image, graphic, and voice). The central 
archive will be significantly expanded, but again no attempt will be made initially 
to handle the entire departmental volume. Performance includes transfer of 2K 
squared by 10 bit images from the central archive to a Q-RSTAR workstation in 
less than one second, and transmission of 512x512 byte by 12 bit images in less 
than 0.05 seconds each. 

MGH Radiology is distributed in various buildings on the main Boston campus 
where radiology currently runs 100 megabit/second fiber optic cable (62.50 - 125 
micron multimode fiber). Additionally, there are five other associated remote 
facilities where we currently run both 45 megabit/sec DS3 and 1.5 megabit/second 
Tl circuits. 
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The table below shows the deployment plan under way for Q-RSTAR workstations 
in addition to the |iRSTAR workstations already in place. 



Location 


Tvne 


MRI@Charlestown 


1 


screen 2K x 2.5K 


MRI @ Charlestown 


i 


_______ t AT/ IV 

screen 1.2K x IK 


To wer-Dodd Room 


2 


screen 2K x 2.5K 


Robbins Room 


1 


screen 2K x 2.5K 


ACC-5 (bone) 


2 


screen 2K x 2.SK 


ACC-2 (chest) 


2 


screen 2K x 2.5K 


White- 1 Emergency Room 


2 


screen 2K x 2.5K 


White-2 Display/Demo Room 


i 
l 


screen zK x z.jK. 


Nuclear Medicuie Interpretation 


1 


screen l.zJv x IK 


Nuclear Medicuie Interpretation 


1 


screen 1.2Kx IK 


White-2 Display/Demo Room 


2 


screen 1.2Kx IK 


Spaulding Rehabilitation Hospital. 


1 


screen 2K x 2.5K 


Suburban HMO. 


1 


screen 2K x 2.5K 


Cambridge RSTAR development center 


1 


screen 2K x 2.5K 


Cambridge RSTAR development center 


1 


screen 2K x 2.5K 


Cambridge RSTAR development center 


2 


screen 1.2Kx IK 



• PACS FIELD TRIALS 

To best understand the PACS activity at the MGH it is best to present it in terms 
of field trials that we will begin late in 1990 and for which we are actively engaged 
in development 

On Demand File-Digitization from Our Main Film Library: 
The MGH Emergency Room and the Ambulatory Care center at the main campus 
frequently request patient images stored in the remote film library located several 
miles away in Somerville. Currently, Somerville is called to request that the 
original images be sent to the main campus on the next shuttle bus which operates 
between the locations hourly during the day. 

This field trial will test the concept of on-demand remote-image-digitization of 
requested images at the remote film library and immediate transmission over a 
broadband network via the department's Q-RSTAR system. Upon entry in Q- 
RSTAR, these images will be available for viewing on any Q-RSTAR 
workstation in the radiology department or as laser printer produced hard copy. A 
number of DuPont FD-2000 210 micron resolution scanners are used that will scan 
an image in 6 seconds. 
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Communicati ons to the Charlestown MRT Out-Patient Facility: 
The Charlestown facility conducts all outpatient MRI studies. Although MGH 
radiologists staff this facility, some MRI studies need to be read at MGH. 
Conversely, radiologists at Charlestown need to access studies conducted at the 
main campus, such as CT. Additionally, there is a need for radiologists at the 
main campus and at Charlestown to consult with each other on a study. 

A DS3 and a Tl circuit connects Charlestown and the main campus. Two GE- 
Signa scanners are in process of being interfaced to Q-RSTAR using TCP/IP via 
Ethernet. Additionally, two Q-RSTAR workstation will be placed at the 
Charlestown facility. The aim of this trial is two-fold; first, to determine whether 
Q-RSTAR can be used as the primary archive for clinical MRI studies, and second, 
to determine if multimedia workstations can be used to effectively 'shrink' the 
distance between the two facilities. Additionally, physicians in the MRI facility 
should benefit from access to other patient images. 

Support to the Spaulding Rehabilitation Hospital: 

MGH provides complete radiological service to this affiliated hospital. Spaulding 
possesses only conventional x-ray equipment Patients requiring CT scans are sent 
to MGH; patients requiring MRI scans are sent to the MGH outpatient facility in 
Charlestown. Both a high resolution workstation and a film digitizer will be 
placed at Spaulding to permit viewing and interchange of studies on Spaulding 
patients at any of the imaging facilities. 

Support to a Suburban HMQ; 

There are many HMOs and imaging centos without proper access to radiological 
services, in particular, radiological sub-specialists. The goal is to investigate 
whether it is possible to yield high quality radiological service by placing a high 
resolution film digitizer at the HMO, digitizing the images, sending the image data 
to MGH via a DS3 circuit, and lastly, providing immediate consultation when 
appropriate and a written report later. If this field trial goes well, a high resolution 
workstation will be installed at the HMO. 



Swport to fts fraflfoiogy Psparwrem; 

The main focus of evaluating RSTAR resides in the MGH Radiology Department 
itself, in spite of the important external activities. The critical question is whether 
the introduction of PACS improves patient care. In this regard the PACS 
development group will be working closely with the MGH Radiologists and 
support staff in incorporating PACS technology into the workings of the 
Department 

It is believed that the Q-RSTAR system will be a solution to the emerging 
problems associated with the existing film based image management systems. In 
particular, the problems due to the increasing volume of image creation, the 
increasing demand for image access, and the requirement to increase retention of 
images. In essence, it is believed that Q-RSTAR will not only be a PACS but a 
work-flow management tool for the department. 
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Calling the above five items 'field trials' does not imply that they are only 
temporary 'experiments'. It is intended that the service will continue if the field 
trial is successful. 

In summary, this is an exciting time for the Department, the hospital, the many 
participating institutions, and especially for the PACS development group that has 
worked over the past five years. 
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1. ABSTRACT 

The Radiology Department at the Hospital of the University of Pennsylvania is 
currently expanding its prototype Picture Archiving and Communication System 
(PACS) into a fully functional clinical system. The first phase of this expansion 
involves three major efforts: the upgrade of the 10 Mbit token-ring to an 80- 
Mbit fiber-optic backbone, the implementation of a large-scale image archive, 
and, the implementation of a software suite called the Folder Manager. The 
Folder Manager has been designed to duplicate and enhance the operation of the 
conventional film library. This paper discusses the functional design of the 
Folder Manager and its role in image acquisition, archival and display. 



2. Introduction 

The Radiology Department has been operating a prototype Picture Archiving 
and Communication System (PACS) since 1986 [1], Efforts are underway cur- 
rently to convert the prototype PACS into a fully functional system over the next 
several years. The first phase of this expansion is designed to meet the archival 
and display requirements of five Magnetic Resonance (MRI) Scanners and three 
Intensive Care Units (ICUs). Details regarding the hardware implementation are 
described elsewhere [2]. In order to duplicate and enhance the operation of the 
conventional film library, a consistent software architecture has been developed. 
It consists of a suite of processes collectively known as the Folder Manager 
(FM). The Folder Manager consists of four functionally separate software mod- 
ules: the Message Manager (MM), the RIS/PACS Translator (RPT), the Data- 
base Manager (DBM), and, the Patient Manager (PM). Each module has been 
designed to duplicate the operation of a specific aspect of the conventional film 
library. It is important to note that while the Master Folder Manager resides on 
the image archive node, a subset of the FM operates at every acquisition and dis- 
play node. The design philosophy of the FM is described elsewhere [3]. 
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Following the discussion in [4], we have developed specific dataflow scenarios 
that describe the interaction among the various components of the Folder Man- 
ager. The dataflows address the following functions: 

• Examination Acquisition using scheduling information from the RIS 

• Examination Acquisition without scheduling information from the RIS 

• Examination cancellation from the RIS 

• Examination Display: Automatic and On-Demand 

We now discuss each of these scenarios in greater detail. The following nota- 
tion is used in the figures: 

FM Folder Manager: a collective term for the MM, DBM, RPT and 

PM 

•MM Message Manager the primary server in the PACS; an information 
router 

•DBM Database Manager 
•RPT RIS/PACS Translator 

•PM Patient Manager responsible for the higher-level optimization of 

the PACS 

RIS Radiology Information System. In our case DECRad 

I ARS The Vortech Image Archival and Retrieval System. The Vortech 
IARS is a large scale image archive capable of storing between 
25GBytes and 7.68 TeraBytes on-line. 

MIG The Vortech Medical Imaging Gateway. The Vortech MIG is a 
computer system designed to provide standard connectivity be- 
tween multiple-vendor acquisition/display devices and a standard 
PACS. 



3. FUNCTIONALITY 

3,1- Image Acquisition using scheduling information from the RIS 
There are three major transactions in this scenario: Examination 
Schedule (denoted by 'a' through V), Image Prestaging(T through 
*iv*), and, Image Acquisition (T through '6'). We now discuss each of 
these in greater detail. 

Examination Schedule: The RIS initiates the transaction by sending an 
Exam Schedule notice to the MM via the RPT ( 4 a\ 4 b*). The MM requests 
and receives image handles from the Image Archive (V). The scheduling 
information is sent to the Image Database (*d'). Finally, the demographic 
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information is sent to the MIG fe'). This is done to decrease operator inter- 
vention when the images are actually acquired. 

Image Prestaging: In general, it is desirable to decrease the volume of 
burst traffic on the network. To achieve this goal the FM estimates the 
number of 'previous' comparison examinations that will need to be viewed 
and transmits them to the display node. Prestaging begins with a trigger 
going off based on an internal timer or patient arrival (T). The MM and the 
DBM communicate with each other to generate the list of previous exami- 
nations (ii). A request is sent to the IARS to pre-stage the images in the 
previous examination(s) (iii). A PREVIOUS.EXAM folder is constructed and 
sent to the display node (iv). Note that the actual pixel data is not transmit- 
ted at this time. 

Image Acquisition: To decrease data-entry errors it is advisable to keep 
operator data-entry to a minimum. With this in mind, the FM sends the de- 
mographics information to the MIG controlling the acquisition device (see 
Examination Schedule above). At the time of acquisition, the technician 
would strobe the schedule notice associated with the examination using a 
bar-code scanner. The barcode scanner would be connected to the MIG ser- 
vicing the acquisition device. Using the accession number read off the 
schedule notice, the MIG would search for the demographics that have been 
previously sent to the MIG (' 1'). If the demographics are found, the opera- 
tor would scan the appropriate image(s) which would be sent to the IARS 
( 4 2'). The IARS then strips the headers from the image and sends them to 
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the MM as ACR/NEMA messages ('3'). The MM collects the images com- 
prising the exam in a folder and adds the exam to the database (*4'). The 
final step is to send the current examination in the form of a folder to the 
display node (*5\ '6'). Note that a possible enhancement in order to expe- 
dite the receipt of exams at the display node is to simultaneously transmit 
image information both to the IARS as well as the display node. 

3.2. iffiagg Acquisition without using scheduling information from tte RIS 
It is not possible to guarantee that the RIS (or the RIS/PACS link) will be 
operational at all times. We now discuss the method for adding examina- 
tions without an operational RIS/PACS link. The MIG does not have the 
demographics information sent to it prior to the examination (see Section 
3.1). Therefore, the operator enters the demographics information and 
scans the images. The images are then sent from the acquisition node to the 
IARS C V). As before, the image headers are stripped and sent to the MM 
( 4 2*). The MM collects the incoming information in holding tables, and 
once the exam is complete, transmits the exam information to the DBM 
('3'). Note that if the DBM has had prior knowledge of the examina- 
tion/patient, the information is considered verified and is entered into the 
database. Otherwise, it is flagged as unverified. If the exam information is 
unverified, an attempt is made to match the information with the RIS ('4') 
and the results of the match are sent to the MM ('5*). If the match failed, 
the MM enques the exam for manual correction ('5*'). If the match suc- 
ceeded, the database is notified. This triggers a current examination folder 
(*6*) which is sent to the display node (*7*). 
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3.3. Examination Cancellation from the RIS 

For a variety of reasons the RIS could cancel a previously scheduled 
examination. To discuss this situation we assume that the events described 
in Section 3.1 have already occurred. Once the cancellation notice is re* 
ceived from the RIS, it is sent to the RPT (* 1 This notice is then forward- 
ed to the DBM via the MM (*2\ '3'). The DBM takes appropriate action to 
delete its entries. 



3.4. Examination Display: Automatic Folder Generation 




If it is possible to predict when and who will view radiological image 
information, it is desirable to be able to send the required images to the 
display node in advance of the viewing session. To enable this 
functionality, the MM has been designed to operate with a rule-base that is 
capable of generating triggers which will initiate a send to the workstation 
(' 1 *). When this trigger occurs, the MM communicates with the DBM to 
retrieve a folder from the database (*2*). The MM then sends the current 
folder to the MIG ('3'). At its convenience, the MIG retrieves the images 
from the IARS ('4'). Note that since the MIG is responsible for managing 
its local store, it is advisable for the MIG to get images rather than have the 
IARS .SSQ& the images to the MIG. After the viewing session, the results 
(which may include the dictated report) are sent to the DBM via the MM 
('5', '6')* Simultaneously, the MM also sends the results to the RIS via the 
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RPT('7\ *8*). 

3.5. Examination Display: On Demand 

The Automatic Folder Generation described in Section 3.4 will not provide 
sufficient functionality for all possible situations. Consequently, the PACS 
should attempt to minimize the radiologist wait-time when an 'on-demand' 
request occurs. The FM does this by analyzing the radiologist's request, 
and making a decision based on the volume of pixel data to be transferred. 

If the FM estimates that the radiologist's request will translate into a large 
volume of pixel data (and consequently a long delay) it queries the database 
and sends a list of exams (and relevant descriptive information) to the 
radiologist with a request to further refine his viewing list Once this is 
done, a folder ('folder by reference') similar to the one described in Section 
3.4 is sent to the MIG associated with the display. The display station then 
retrieves the image information. We now discuss a possible scenario in 
greater detail. 

The radiologist requests an exam for viewing (* 1 '). The request is forward- 
ed to the DBM via the MM (*2\ *3'). The DBM responds with a list ( 4 4*) 
which is returned to the MIG via the MM (*5'). The MIG forwards the list 
to the display station with a request to the radiologist to further narrow 
down the selection ('6'). The radiologist makes the selection which is 
passed to the MIG ('7'). The MIG retrieves images for display (*8'). 




248 



4. Summary 

A software suite called the Folder Manager has been designed and implemented 
on the Vortech IARS. The goal of the Folder Manager is to optimize the 
acquisition, communication and display of radiological images on a PACS. The 
software is currently under B-test 
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Introduction 

A Level 1 Trauma Center creates an intense medical 
environment targeted at a limited spectrum of high risk 
disease. In the trauma intensive care setting a digital 
imaging system must satisfy the requirements for 
radiography of the chest, since twenty-five percent of 
trauma-related deaths can be attributed to chest injuries. 
This study was undertaken to determine whether digital 
radiography is of sufficient diagnostic accuracy, in the 
Trauma Intensive Care Unit (TICU) setting, to replace our 
current analog system. The chest film examination was 
selected as the subject of the study because of its 
importance to the TICU patient. 



Materials and Methods 

Acquisition of the study population took place over a 
three month period from January, 1990 through March, 
1990. All images for interpretation were acquired from the 
TICU patient population of the MED using portable 
radiography. The MED has a maximum capacity of eighteen 
TICU beds. Each day, three patients were filmed on both 
the analog and digital systems. No patient was rest u died 
until the entire remaining population of the TICU was 
included in the study. 

All digital examinations for this study were acquired 
using a Siemens Digiscan Computed Radiography system. 
The digital images were exposed on 14"x17" ST3 cassettes 
and were interpreted from film at a 2:1 reduction factor. A 
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second processed image of the digital study was also 
available for viewing on the same film. 

The conventional and digital examinations were both 
exposed with radiographic parameters considered optimal 
for the conventional study. If the conventional examination 
was considered technically inadequate by the interpreting 
radiologist or the technologist, that set of radiographs was 
eliminated from the study. 

The study attempted to simulate the normal 
circumstances of film interpretation for our TICU. No clinical 
histories were provided other than the knowledge that these 
were all TICU patients. No limitation in diagnoses was 
imposed on the readers. A complex form was designed to 
cover all diagnostic possibilities on chest examinations and 
the readers completed this form during or immediately after 
their interpretation of each radiograph. 

A brief explanation of the contents of the form was 
provided each radiologist before their first reading session. 
An additional brief introduction to the concept of reporting 
images appropriately for ROC analysis was provided for 
each reader. A total of six board certified radiologists read 
each study. Three radiologists read the conventional 
examinations first and the other three read the digital 
examinations first. 

At the completion of all readings, all six involved 
radiologists convened and reviewed the combined digital 
and conventional films as a panel of experts. Consensus of 
the panel on each feature of the examination constitutes 
truth for the purposes of this study. 

Results were analysed using a chi-square analysis. 
Because of the small number of patients in some cells, 
Yate's correction was applied when the p value was 
calculated. 

Results 

Fifty-six conventional and fifty-six digital chest 
examinations obtained on thirty-seven patients were each 
interpreted by six board certified radiologists. In all cases, 
this was the first opportunity for the radiologists to interpret 
digital radiographs. 
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The film interpretations were divided into positive and 
negative findings. Inconclusive interpretations, "possibly 
present" and "possibly not present", were not included in 
either the positive or negative groups. The results are 
summarised in Table 1 . There was no significant difference 
between the interpretation of the analog and digital films in 
all categories except for mediastinal hematoma. In this 
category, analog films were superior to digital ones. A more 
detailed statistical analysis will be presented. 

Table 1 



Abnormality 


Conventional 

(+) ve (-) ve 


Digital 

(+) ve (-) ve 


p Value 


Air space disease 


173 


103 


190 


84 


n.s. 


Interstitial 


102 


158 


86 


154 


n.s. 


Mixed pattern 


49 


191 


55 


177 


n.s. 


Mass 


6 


298 


2 


285 


n.s. 


Emphysema 


7 


299 


9 


285 


n.s. 


Mediastinal hematoma 


21 


244 


9 


255 


<0.04 


Effusion 


89 


141 


83 


142 


n.s. 


Pneumothorax 


12 


288 


8 


280 


n.s. 


Pneumomediastinum 


9 


306 


8 


316 


n.s. 


Spinal fracture 


5 


147 


4 


165 


n.s. 


Rib fracture 


87 


201 


84 


187 


n.s. 


Hilar adenopathy 


1 


264 


1 


253 


n.s. 


Mediastinal adenopathy 


1 


256 


0 


260 


n.s. 



Discussion and Conclusions 

This study represents an attempt to evaluate the 
effectiveness of digital radiography in the "true-to-life" 
clinical setting in which a radiologist evaluates all possible 
findings on a radiograph, rather than focusing on selected 
findings thought to test the resolution of the system. All 
radiologists involved in this study were considered novices 
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in regard to digital radiography and the study design itself is 
felt to be extremely arduous by comparison to a study that 
evaluates only one or two features, such as pneumothorax. 

The data presented is considered to be preliminary data 
from a much larger and ongoing study in progress. 

This preliminary study demonstrates that for the most 
part, digital films are comparable to analog films. However, 
the presence of mediastinal hematoma was recognized 
less frequently on digital films. While this deficiency may 
have been caused by the limited experience of the 
investigators, it may reflect an area in which analog films 
are superior to digital films or may be caused by the 
reduced size of the digital image compromising estimation 
of mediastino-thoracic ratio. Later results from our ongoing 
study are expected to determine whether digital 
radiographs are accurate enough to replace analog 
radiographs in a TICU. 
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ABSTRACT: 

Prototype electronic workstations incorporated in 
networks linking computed tomography and magnetic 
resonance imaging systems are being developed- A 
comparative evaluation of observer efficiency and 
sensitivity in reading body CT studies from a two- screen 
workstation (1000 line monitors and 12 bit dynamic range 
in image memory) and conventional film panel alternator 
was performed. The two-screen workstation displayed 32 
images at 256^ matrix resolution or 8 images at 512^ 
matrix resolution simultaneously. Ninety-six images at 
512 2 matrix resolution could be displayed simultaneously 
at the film panel alternator. Four observers read 20 case 
studies, 10 with repeat examinations, in a randomized 
viewing sequence. There was an average of 32 images per 
case. Reporting time on the film panel alternator was 
faster (Film Panel Alternator - average = 5.07 minutes, 
workstation - average time = 6.66 minutes). Extra 
analysis time at the workstation was required if observers 
analyzed cases initially at 256^ resolution and 
subsequently at 512 2 resolution. There was improved 
sensitivity by all observers in reading from the Film 
Panel Alternator (range l-12%)(p <0.05). 

In reading complex body CT cases, the current 
prototype two-screen electronic workstation is limited by 
display capabilities. 



Reprinted from "RADIOLOGY, " Vol- 174, 
March 1990, Pgs. 769-773. 
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INTRODUCTION: 

Multiscreen workstations linked via network to 

primary imaging systems and with on-line and off-line 

electronic storage are projected to be the diagnostic hub 

of a Radiology Picture Archiving and Communications System 

(PACS). While the resolution requirements for the 

electronic display and diagnosis of radiographic images, 

particularly chest radiographs, have been extensively 

studied , requirements for the accurate and efficient 

reporting of multiple lower resolution cross-sectional 

images from a multiscreen display station have not been 

addressed- While a total radiology PACS is not practical 

unless digital radiography can be implemented effectively 

from an economic and diagnostic viewpoint, mini-PACS 

linking CT and MRI systems to a multiscreen workstation 

are a near term possibility. A workstation can be used 

for on-line monitoring and review of current studies from 

multiple imaging systems. In addition, the workstation is 

a potential replacement of the film panel alternator for 

study reporting and, via connection to an electronic 

archive could lead to the replacement of film as the 

archival medium for CT and MRI studies. 

In this study, we evaluated the capabilities of a 

prototype two-screen electronic workstation for reporting 

and based on these initial results suggest modifications 

and improvements to the current prototype system. 

Materials and Methods: 

Twenty body CT studies which were performed in 1987 
and for which either a tissue or conclusive clinical 
diagnosis was available were chosen. Cases were selected 
from consecutive studies beginning at a fixed time point 
(June 1, 1987) until a representative series of studies 
related to trauma, inflammation, vascular disease, 
detection and staging of malignancy and diffuse organ 
disease were available. Selected cases demonstrated focal 
or multifocal pathology in the neck, thorax, abdomen and 
pelvis and contained an admixture of readily perceived and 
subtle abnormal findings varying in size from 5 mm 
diameter (focal lung lesion) to several centimeter masses 
(focal pancreatitis, hepatic metastases, pulmonary 
metastases, abdominal aortic aneurysm, et cetera) to 
diffuse organ disease (e.g. cirrhosis). The cases were 
typical of those seen in a tertiary care medical center 
performing 7000 body CT exams per annum. There were two 
cervical, two thoracic, two thoracic/abdominal, six 
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abdominal , seven abdominal/pelvic and one pelvic CT 
studies. The diagnostic findings , for each anatomic area 
are outlined in Table I. In ten of the patient studies, 
two CT examinations were performed. In each of these ten 
cases, both the images and diagnostic impression from the 
initial CT study were pertinent to interpretation of the 
second examination. The list of diagnostic findings, both 
pertinent to patient management and incidental for each 
study was established by an expert panel who reviewed all 
the CT images and the pertinent clinical information and 
tissue diagnoses, if available. The expert panel did not 
relate any of the selection criteria or specific diagnoses 
to the observers who participated in the experiment. 

The cases were read by four experienced 
radiologists who had acquired familiarity with the 
operation of the electronic workstation by reviewing 
studies from a remote imaging system and/or transmitting 
selected images from the electronic workstation to a 
physicians viewing station elsewhere in the hospital. All 
four radiologists were experienced body CT readers, but 
had not previously diagnosed studies from the two-screen 
display station. The readers analyzed each case from the 
two-screen electronic workstation and from a film panel 
alternator. Each reader was provided with a pertinent 
clinical indication for CT study, equivalent to what would 
be expected during normal radiology reporting. To 
approximate the daily working environment as closely as 
possible, the descriptive report and impression were 
dictated onto an audio cassette and subsequently 
transcribed. 

For the study format, each reader initially read ten 
cases from the film panel alternator and the other ten 
cases from the electronic workstation. For the film panel 
reading, each case was displayed on a 12/1 multiformat 
film with a standard window level settings appropriate for 
neck, thorax or abdominal studies. Thoracic studies were 
also filmed at "lung windows." Cases with skeletal 
pathology (e.g. vertebral metastases ) had images displayed 
at bone windows. These studies were arranged on a 50 
panel alternator which allowed two panels to be viewed 
simultaneously, each panel allowing four films or 48 
images to be displayed. Thus, a reader had direct access 
to 96 512 2 matrix CT images without manual change of the 
film panels. Each neck, thorax or abdominal study could 
be read from one film panel. In the ten studies in which 
progress thoracic or abdominal CT examinations were 
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performed , both the preliminary and subsequent CT study 
could be evaluated on the two panels simultaneously. For 
the thoracic/abdominal CT studies, the thoracic or 
abdominal sections of the two studies could be compared 
simultaneously, but not both together- All patient 
identification marks on the film display were covered. 

The electronic workstation utilized two video 
monitors, each 1280 horizontal lines operating in a 60 
Hertz non-interlaced format with a bandwidth of 78.6 MHz 
providing a 1024 x 1024 image display (General Electric 

Medical Systems, Milwaukee, Wisconsin). The video 
monitors were 19" oblique diameter and displayed menu 
options and patient identification at the upper and lower 
margins of the screen. The image data was displayed using 
a dynamic range of eight bits with an interactive window 
width and window level function allowing evaluation of the 
full 12 bits of image data. 

Screen luminance for a water density object (using 
a window level of 0 and window width of 350) was 55 cd/m 
for the film panel display and 6.2 cd/m 2 for the video 
screen (Fig. 1). Screen luminance was linear over a wide 
range of tissue densities for both viewing modalities 
except for the more transmitting components of the image 
(e.g. contrast material and bone) (Fig. 1). The film 
display had a slightly greater contrast than the video 
screen. The screen illuminance (the light level incidence 
on the screen from surrounding room lights) was 
approximately 33 lumen/m 2 at the average ambient light 
levels employed in the experiment. 

At the electronic workstation, 512 images were 
displayed 4/1 on each screen and 256 2 images were 
displayed 16/1 on each screen. The video screens could be 
operated in a "merge 11 mode allowing 32 consecutive images 
from one study to be displayed. Alternatively, with the 
"split screen" mode, different studies of the same patient 
could be displayed on each screen and compared. 
Instantaneous window level controls to preset values for 
soft tissue display of the neck, mediastinum and abdomen 
and for display of lung parenchyma and skeletal structures 
were provided. The window width and window level display 
values were equivalent to those utilized for film display 
of these regions. Additional window and level changes 
could be made by each reader using a "mouse" control. 
Thus, each reader could interactively change contrast 
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throughout the full dynamic range of the video display , 
while the range of contrast displayed on the film images 
was fixed. 

Cases were selected from a displayed list using the 
interactive mouse and menu screen prompts. Screen display 
times for eight 512 2 images was 7.5 seconds and for 32 
256^ images was 26 seconds. Cases to be displayed at the 
electronic workstation were preloaded onto a 600 Megabyte 
magnetic disk incorporated in the workstation. 

Four weeks after the initial two reading sessions 
(10 cases read from the film panel alternator in one 
session and 10 read from the electronic workstation in the 
other), the cases were reread from the alternate viewing 
modality using a different random sequence. Thus, each 
observer participated in four reading sessions. Reading 
time varied between 45 and 90 minutes dependent upon 
reader speed. No time constraints were put on the 
individual reader. 

The dictated reports were analyzed for the readers 1 
sensitivity in detecting abnormal findings in each of the 
studies. The diagnostic findings in the dictated reports 
from the film and video readings were tabulated and 
compared to the expert panel findings to determine 
sensitivity for each observer for each viewing modality. 
The abnormal findings were subdivided into two categories; 
namely, all abnormal findings and secondly, the clinically 
significant abnormal findings pertinent to patient 
management. In addition, false-positive diagnostic 
findings were tabulated and analyzed. For each abnormal 
diagnostic finding as listed in the impression, readers 
provided a subjective confidence level on a rating scale 
between 1 and 10, with 10 being the most confident. 
Differences in readers 1 sensitivity in reporting both 
clinically significant and incidental diagnostic findings 
were compared using the paired two-tailed student T Test 
for analysis of discrete data sets. Subjective confidence 
levels were compared using Chi-Square analysis. 

The film and video reading sessions were analyzed 
for time involved in image analysis and dictation for each 
reader. Image analysis time was from the reading of 
"indication for study" until the beginning of dictation. 
Dictation time was from the beginning of the audio 
dictation until completion. Each observer continued image 
analysis during dictation. For the video reading 
sessions, the number of screen changes, window level 
changes, review time for reanalyzing a previous study when 
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reporting the current examination and time required to 
write pertinent diagnostic impressions before completion 
of image analyses was recorded. Screen changes recorded 
two events: a) changing from the 16/1 format to 4/1 
format , and b) paging through an examination using either 
screen display format. Each individual was also assessed 
for overall approach to diagnosis from the electronic 
workstation , specifically whether 256^ resolution with 
16/1 display format was used or whether a combination of 
256 2 and 512 2 resolution with 16/1 and 4/1 display format 
was used. 

At the end of the study , each peader was interviewed 
to obtain subjective impressions on the presence or 
absence of flicker and mental or eye fatigue , ease of 
comparison using the film and video display systems and 
what additional improvements each reader might consider 
for the electronic workstation. 

Results : 

The sensitivity of all readers improved when 
reporting from the film panel alternator as compared to 
reporting from the two-screen workstation (Table I). The 
improvement in sensitivity varied between 1 and 12%. For 
all positive findings, the improvement in sensitivity 
varied between 3 and 12% and for significant findings 
between 0 and 8%. The difference in sensitivity between 
film and video reading was significant at a P value (<) 
,0.05. 



TABLE I 



SENSITIVITY 



RADIOLOGIST 



VIDEO 



FILM 



A 



84 (91) 



96 (98) 



B 



79 (84) 



82 (90) 



C 



74 (81) 



82 (85) 



D 



76 (85) 



80 (85) 



Findings Only 



( ) Significant 
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TABLE II 
VIDEO READING 





A 


B 


C 


D 


NO. OF SCREEN CHANGES 


2.63 


17.28 


9.77 


9.55 


NO. OF W/L CHANGES 


3.20 


2.50 


2.00 7.86 


PREVIOUS STUDY REVIEW 










(MINUTES) 


.08 


2.75 


1.14 


1.29 


NfYPI? UD TTTNn / liTWTFFT? Q \ 
fHUJ.iL nAlllnu inXfiU 1J1D J 








.09 


ANALYSIS (MINUTES) 


2.37 


7.17 


3.67 


3.67 


DICTATION (MINUTES) 


2.04 


2.31 


2.70 


2.70 


TOTAL (MINUTES) 


4.41 


9.48 


6.37 


6.37 



All readers had an improvement in subjective 
confidence level when reporting from film as compared to 
the two-screen video console. However, this improvement 
was significant , as analyzed by the Chi-Square test, only 
or Reader A. 

Pertinent findings which were missed on the video 
display but diagnosed from the film display included 
pericardial effusion in a patient with recurrent breast 
carcinoma (missed by one observer) and a 3 cm soft tissue 
lesion adjacent to the left piriformis muscle in a patient 
with a primary rectal carcinoma (missed by one observer). 
Pertinent findings which were missed on both the video 
and film display included a cecal carcinoma in a patient 
with multifocal hepatic lesions (missed by one observer), 
left adrenal enlargement in a patient with melanoma (two 
observers), left adrenal mass in a patient with carcinoma 
of the lung (one observer) and cirrhosis in a patient with 
abdominal aortic aneurysm and aortic dissection (two 
observers). An incidental finding which was missed by all 
observers on both video and film display was a subscapular 
lipoma in a patient with a history of malignant fibrous 
histiocytoma and a small pulmonary nodule. 

There were nine false-positive observations, only 
one of which was made by more than one observer. Four 
false-positive observations were made on both the video 
and film display and the other five only from the video 
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display. For each viewing modality, there was a total of 
120 separate study reports (30 study reports - 20 cases, 
10 of which had comparison examinations - and 4 readers) 
containing 160 true-positive observations. Only one 
false-positive finding, (pneumoperitoneum from blunt 
abdominal trauma), which was made from the video display 
would have altered patient management. 

There were marked differences in analysis time when 
comparing the film and video reading formats (Table II). 
The average analysis time for film reading was 2.56 
minutes (range 1.62 to 4.42 minutes) and for video reading 
4.22 minutes (range 2.37 to 7.17 minutes). Reader A was 
the fastest film analyzer (1.62 minutes) and also the 
fastest video analyzer (2.37 minutes). Conversely, Reader 
B was the slowest film analyzer (4.47 minutes) and also 
the slowest video analyzer (7.17 minutes). Reader A 
reviewed all video cases at 256 2 resolution only (16/1 
display format) with minimal screen changes. Reader A did 
not use visual side-by-side comparison with the split 
screen mode when comparing two CT studies done at 
different times. Readers C and D reviewed all video cases 
at 256 2 and subsequently 512 2 resolution (4/1 display 
format). Average analysis time was 3.67 minutes. A 
proportion of this analysis time involved use of the split 
screen mode for visual comparison of CT images when 
comparing two CT studies. If the cases were analyzed in 
a very detailed fashion by review at 256 2 and subsequently 
512 2 resolution and paging forwards and backwards through 
an individual case using the 5 12 2 format, total analysis 
time increased proportionately as evidenced by Reader B 
(7. 17 minutes) . 

Except for Reader D, dictation time for film reading 
and video reading was comparable. Average dictation time 
for film reading was 2.51 minutes and for video reading 
2.70 minutes. 

Total analysis and dictation time for film reading 
averaged 5.07 minutes (range 3.92 - 7.37 minutes) and for 
video reading averaged 6.66 minutes (range 4.41 - 9.48 
minutes ) . 

For all readers, there was decrease in efficiency 
when reading from the video display versus the film 
display. This decrease, as a percentage of the total 
reading time for film reading was 13% for Reader A, 29% 
for Reader B, 19% for Reader C and 70% for Reader D. 
Reader D had the shortest total reading time for film 
reading (3.75 minutes per case), explaining the greater 
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percentage decrease in efficiency of this particular 
reader. However, Reader D was intermediate in efficiency 
in reading from the two-screen video display (6.37 minutes 
per case) . 

Discussion: 

For all readers , lesion detectability was inferior 
when reading from the film two- screen video console as 
compared to the film panel alternator. This improvement 
ranged from 3-12% for all abnormal findings and between 0- 
8% for significant abnormal findings and was statistically 
significant. False-positive observations were rare for 
all readers. The minimal false-positive observations 
probably relate to study design in that each observer was 
provided with a pertinent clinical history consistent with 
the abnormalities present on each of the CT studies. 

Reader A was the fastest reader from video and film. 
This reader consistently used only the low resolution mode 
of the video display (256^ resolution - 16/1 format). 
Somewhat paradoxically, this reader also was the most 
accurate reader from the video display both for all 
abnormal findings as well as clinically significant 
abnormal findings. However, this reader also had the 
greatest percentage improvement in lesion detectability 
when comparing video reading to film reading. Reader A's 
improvement in accuracy appears more directly related to 
image resolution limitations of the 256^ Matrix 
presentation. The other three readers routinely evaluated 
CT studies at the video console by initially surveying 
each case using 256 Matrix (16/1 display format) and then 

subsequently re-reviewing each case using the 512 Matrix 
(4/1 display format). For these three observers, failure 
to record positive abnormal findings probably relates to 
either failure to detect these abnormalities on the video 
and/or film display or perhaps failure to memorize the 
abnormality while dictating the case as another segment of 
the anatomy was displayed on the video screen. The lower 
luminance of the video display in comparison to the film 
panel alternator"^ as well as minimal flicker in the video 
display could also have affected the results. 

Reader A, using the 16/1 display format was able to 
view 32 images simultaneously from the two-screen video 
workstation and it is not surprising that Reader A had the 
shortest video reading time and the least percentage 
increase in reading time in comparison to the film panel 
alternator. For readers B, C and D, the major component 
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of the increased analysis time for video readings was the 
greater number of screen changes, predominantly paging 
through studies using the 4/1 screen display format. 
Reader D used many more window width and window level 
changes than the other three readers, though this 
instantaneous on-off function did not result in increased 
analysis time. 

This study indicates that for all readers 
sensitivity and efficiency in reading from the video 
display station were inferior to reading from the film 
panel alternator. The individual differences between 
readers also demonstrate individual variation that affects 
the performance evaluation of these viewing modalities. 
Reader A who presumably has better perception and memory 
than the other readers performed better at the video 
display than some of the other readers on the film 
display. Nevertheless, Reader's A performance when 
reading from the film panel alternator was still better 
than when reading from the video display. All readers were 
unanimous in suggesting that for complex body CT cases an 
electronic display station would be improved by using four 
screens rather than two. All readers would prefer a 
higher resolution video display (e.g. 2048 x 2048 or 2048 
x 1500), but for the 19" screen size used in this study 
suggested a 12/1 display format at 512^ matrix would be 
preferable. Use of the 12/1 display format would maintain 
image size comparable to that currently utilized for 
multiformat film recording. All readers, including reader 
A, felt that image resolution using the 256 2 matrix is a 
limitation. Readers indicated that it is more difficult 
to remember perceived abnormalities while dictating if the 
appropriate images are not on the display while the 
dictation is being performed. The subjective feeling of 
mental and eye fatigue appeared directly related to the 
total analysis time for reading the video display. When 
exam comparisons were made, all readers indicated that 
this was performed better with film rather than the video 
display as 96 images could be displayed simultaneously on 
a film panel alternator and only 32 (16 per individual 
study) on the split screen video display. 

A workstation linked to CT/MRI imaging systems has 
the potential to allow one radiologist to simultaneously 
monitor more than one imaging system and to dictate 
complex cases "on-line." We do not feel that our 
prototype system allows radiologists to dictate complex 
body CT cases with sufficient accuracy and prefer to use 
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the conventional film panel display for this purpose. 
Improvements in faster image display writing time, and 
total number of images displayed at full resolution should 
be beneficial. In essence, what these radiologists seemed 
to want for body CT reporting is an electronic workstation 

that simulates a film panel alternator as closely as 
possible. The alternative option, which was not evaluated 
in this study, would be to provide radiologists with a 
visual cue using miniaturized reference images that would 
be available as a "memory refresh" during dictation if 
the complete study could not be displayed at full 
resolution. 

We plan to carry out a similar study using Magnetic 
Resonance case studies as images can be displayed at full 
resolution using the 16/1 display format. Although there 
are lesser requirements for display resolution for MRI as 
compared to CT, there are also a greater number of images 
per study. 

In the future, it is likely that an appropriately 
designed electronic workstation linked to more than one 
imaging system could replace the physician viewing 
consoles of those imaging systems and could allow 
radiologists to improve efficiency by the use of on-line 
dictation from these devices. However, improvements in 
operational performance, as suggested by this study would 
seem necessary before this could be a practical 
implementation for complex body CT cases. 

REFERENCES 

1. Seeley CW, Fisher HD, Stempski MO et al. Total 
digital radiology department: spatial resolution 
requirements. AJR 1987;148:421-426. 

2. Lams PN, Cocklin ML. Spatial resolution 
requirements for digital chest radiographs: ROC 
study of observer performance in selected cases. 
Radiology 1986;158:11-19. 



265 

MacMahon H, Vyborny CJ, Metz CE, Doi K, Sabegi V, 
Solomon S. Digital radiography of subtle pulmonary 
abnormalities: ROC study of the effect of pixel size 
on observer performance. Radiology 1986;158:21-26. 
Goodman LR, Foley WD, Bolson CRT, Rim AA, Lawson TL. 
Digital and conventional chest images: observer 
performance with film digital radiography system. 
Radiology 1986;158:27-33. 

Goodman LR, Foley WD, Wilson CR, Tikofsky, RS, 
Hoffmann RG. Pneumothorax and other lung diseases: 
Effect of altered resolution and edge enhancement on 
diagnosis with digitized radiographs. Radiology 
1988;167:83-88. 

Foley WD, Wilson CR, Keyes GS, et al. Effect of 
varying spatial resolution on the detectability of 
diffuse pulmonary nodules. Radiology 1981; 141: 25- 
31. 

Foley WD, Wilson CR, San Dretto M. The effect of 
varying high pass filter parameters on the 
detectability of isolated pulmonary nodules in 
digitized chest images. Investigative Radiology 
1986;21:305-310. 



266 

8. Huebener KH. Scanned projection radiography of the 
chest versus standard film radiography: A 
comparison of 250 cases. Radiology 1983 ; 148: 363- 
368. 

9. Kushner DC, Cleveland RH, Herman TE, et al. Low 
dose flying spot digital radiography of the chest: 
Sensitivity studies. Radiology 1987;163:685-688. 

10. Kundel HL. Digital projection radiography of the 
chest. Radiology 1986;158:274-276. 

11. Fraser RG, Breatnach E, Barnes GT. Digital 
radiography of the chest: Clinical experience with 
a prototype unit. Radiology 1983;148:1-5. 

12. Foley WD, Goodman, LR, Wilson CR. Film digital 
radiography system: Effect of pixel size and image 
processing on the detection of interstitial lung 
disease in "proceedings of the Chest Imaging 
Conference"; Madison, Wisconsin, September, 1987:96- 
109. 

13. Macmahon H, Vyborny C, Sabeti V, Metz C, Doi K. The 
effect of digital unsharp masking on the 
detectability of interstitial infiltrates in 
pneumothoraces . Proceedings of SPIE 1985; 555: 246- 
251. 



267 



14. Kundel HL. 
terminals . 
1986;24:69. 



Visual perception and image display 
Radiologic Clinics of North America 



First Phase of the Dutch PACS Project (1986-1989): 
Research Outcomes and Clinical Evaluation Results 

Fenno R Ottes, Albert R. Bakker and Jef M.L. Kouwenberg 
BAZIS, Central Development & Support Group Hospital 
Information System, Leiden, THE NETHERLANDS 

Bart M. ter Haar Romeny 

Radiology Department, Utrecht University Hospital 

Utrecht, THE NETHERLANDS 



ABSTRACT 

In the period from 1986-1989 a PACS project has been carried out in the 
Netherlands. It comprised research, software development and clinical evaluation 
of a prototype PACS. The project was jointly performed by: 

- the Utrecht University Hospital (UUH), 

- BAZIS, Central Development and Support Group HIS, Leiden, and 

- Philips Netherlands/Philips International product division Medical Systems. 
The research and development activities were conducted by the UUH and 
BAZIS and funded by the Netherlands Ministry of Health Care. The prototype 
PACS system was provided and maintained by Philips. 

This paper concisely reports on the activities in this project, and the results 
obtained. The major conclusion is that the implementation of a PACS can 
improve the logistics of the diagnostic image process in a hospital. It has been 
shown to be essential for the successful operation of a PACS, to couple it with 
the existing HIS, that must contain certain image related subsystems, e.g. an 
RIS. Simulation promises to be a good tool to support the design of the PACS 
architecture. Furthermore, diagnostic image quality of the prototype must be 
improved to be able to use it for primary diagnosis. 

Based on a calculation of the costs of the prototype it has been estimated that 
hospital-wide PACS in the UUH would at this moment be much more expensive 
than the existing film-based system. In future however PACS costs can be 
expected to decrease. Concrete recommendations for future PACS 
implementation research are stated based on our experiences in this project. 

The project is described in full detail in the, recently completed, 140-page 
concluding report [1]. 



INTRODUCTION 

Shortly after the introduction of the optical disk technique the first PACS 
research project in the Netherlands was started in the early eighties. This pilot 
study on Image Storage Management was conducted by BAZIS and subsidized 
by the Dutch Ministry of Health Care. 
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This research led to the set-up of a grant proposal for the on-hand PACS 
research project by UUH, BAZIS and Philips. The target of this applicability 
study was to obtain more insight into the clinical aspects of PACS. The study 
was focussed on the potential benefits and how PACS should be implemented to 
obtain these benefits. 

A multi-disciplinary approach is advisable to overcome the complexity of the 
PACS concept. The expertise from various disciplines should be combined and 
the partners should mutually stimulate each other: 

• A hospital should provide the clinical test bed and the knowledge on the 
diagnostic process needed to perform a PACS prototype evaluation. 

- An industry should furnish the PACS equipment. It should also allow 
feedback of the study results to the product development. 

- A research group, that knows the information requirements in a hospital and 
is independent from the PACS vendors, is able to conduct an independent 
evaluation of the PACS system performance in a clinical environment. The 
obvious disadvantages of a blunt technology push can be avoided in this way. 

- A coupling with the existing HIS should be realized. 

To establish an optimal contact with the Dutch radiologists in general the Dutch 
Society of Radiodiagnostics was invited to closely follow all phases of the project. 

The main objective of the Dutch PACS project was, as mentioned in the grant 
proposal: to achieve a presentation of images coupled with related patient data, 
that allows the radiologist (and the requesting physician) to perform flexible, fast 
and fruitful diagnostic work. 
The various activities that were conducted are: 

- Coupling and integration of the Philips prototype and the BAZIS HIS, 

- Modelling and simulation of PACS hardware and its use in the hospital, 

- Diagnostic image quality evaluation, 

- Technical evaluation of the prototype PACS, 

- Evaluation of the PACS coupled with the HIS used in clinical routine, 

- Cost and benefit study of the prototype and for a whole hospital. 
These studies are described in the next section. 

The research partners can be characterized as follows: 

- The University Hospital Utrecht is a teaching hospital with 900 beds, 3500 
employees and 110,000 radiological examinations per year. Almost all 
imaging modalities are available: MR, CT, DSA, US and NM. The BAZIS 
HIS with about 800 connected terminals, is installed in the UUH. Until July 
1989 the UUH was housed in a conglomerate of old free-standing buildings 
in the center of Utrecht. At that date it has been moved to a single new 
building at the periphery of the city. 

- BAZIS develops and maintains the BAZIS HIS, an integrated hospital 
information system that contains 70 subsystems. The RIS subsystem is fully 
integrated with all other subsystems and includes reporting, appointment 
scheduling, and archive management and lending functions. The BAZIS HIS 
has been developed since 1972, and is currently used in more than 40 Dutch 
hospitals (together over 19,000 acute beds). 

The proposal was accepted and funds were allocated to a total amount of fl. 4.4 
million for the full 3-year project period. 
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ACTIVITIES AND RESULTS 

The clinical evaluation took place primarily at one single ward (15 beds) of the 
Internal Medicine department During the evaluation period, all relevant 
radiological images were provided in digital form for all patients at that ward. 
The patient related data were made available together with the images on the 
PACS workstation by means of a coupling between the HIS and the PACS. 

COUPLING/INTEGRATION HIS/PACS 

From the very beginning of the project it was recognized that a coupling 
between the prototype PACS and the existing HIS is necessary [2]. Many of the 
user demands concerning hospital data integration, PACS workstation 
functionality and PACS response times can only be met when such a coupling 
exists: 

- for the identification of the images HIS patient data is required. The re- 
entry of this HIS information, and the errors that certainly will occur doing 
that, can be avoided by sending the data through the HIS-PACS coupling. 

- together with the images related alphanumerical patient information from the 
HIS, including patient demographics and medical data like radiological 
reports and diagnoses, must be displayed on the PACS workstation in a 
coherent way. 

- acceptable waiting times for PACS systems can with the current technology 
only be achieved when intelligent image management strategies are 
implemented. For example prefetching, i.e. to reduce waiting times images 
should be transferred from the slow archive to the fast workstation memory 
some time before the user actually requests to display these images. The 
image management system can be fed by HIS information, such as 
appointment and patient tracking data. 

Actually a one-way PACS-HIS coupling was realized at the UUR The record 
format of the messages was based on the ACR/NEMA standard. The message 
transfer was triggered by the registration in the HIS of certain events in the 
hospital. Such events were: admission, transfer or discharge of a patient, 
appointment made, changed or cancelled for a radiological examination, and 
authorization of a radiological report. The transferred information consisted of 
patient demographics and radiological history and radiological reports [3]. 

The coupling was first demonstrated during the 6th EuroPACS meeting that was 
organized in Utrecht and Leiden on April 25 and April 26, 1988, respectively. 
The coupling was in use during the full duration of the clinical evaluation (see 
section below). All transferred data has been logged and the log files have been 
used for analysis of tne performance. 

MODELLING AND SIMULATION 

As we started the PACS project real-life experiments with hospital-wide PACS 
systems could not be performed. Such systems were not yet available and would 
be very expensive. Implementation of (hospital-wide) PACS should be well 
prepared, because mistakes in an early stage could lead to a large set-back in the 
development and implementation of PACS systems. 
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For these reasons, which are now still valid, we decided to use modeling and 
simulation in PACS research. By creating a model of the PACS combined with 
its hospital environment and by performing simulations of that model, the design 
of the PACS hardware can be supported. Also the adaptation of clinical working 
methods can be guided to allow an optimal implementation of PACS. 
A simulation package, called MIRACLES (Medical Image Representation, 
Archiving and Communication Learned by Extensive Simulation) has been 
developed. This package has been used to analyze the performance of existing 
and imaginary PACS systems, including various models of the Philips PACS 
product line [4], Simulation run results clearly demonstrated that prefetching is 
essential to be able to create PACS systems with acceptable waiting times. 

DIAGNOSTIC IMAGE QUALITY and USER INTERFACE 
A crucial condition for the acceptance of PACS by the radiologists is the 
diagnostic image quality of such systems. The commonly advocated method for 
evaluation of diagnostic accuracy is the observer performance study, that 
includes the Receiver Operating Characteristic (ROC) technique for the 
statistical data analysis. 

A PC software package to support all phases of ROC studies has been 
developed. The package, which is called FEASIBLE (Feature Evaluation And 
System Inspection By Logged Experiments), allows the user to perform ROC 
studies in an easy and controlled way. It aids in the design of the experiment as 
a whole, in the execution of the psychophysical sessions and in the calculation of 
the ROC curves. Figures ready for publication, that contain multiple ROC 
curves, can be plotted with the package. The package has been sent, upon 
request, to over 25 interested research groups. 

The applications of FEASIBLE include evaluations of the Philips prototype. It 
has been studied whether the diagnosis, based on an image digitized with the 
laser scanner and presented on a monitor of the Philips workstation, has the 
same quality as a diagnosis based on the original film viewed on a light box. The 
result largely depends on the kind of images that are used. For thorax phantoms 
and clinical mammograms it was shown that the image quality of the PACS 
system applied in the project is insufficient For clinical head CTs no significant 
difference was detected [5]. 

Another important issue for the radiologist's acceptance of PACS is the human 
interface of the image workstation. Software for the retrieval, the display, and 
the manipulation of radiodiagnostic images on an image workstation based on 
general purpose hardware has been developed [6]. This package, called 
FRACTALS (Filmless Radiology and Computerized Task Analysis Lab 
Software), has been applied in research on image enhancement methods. The 
built-in journalizing facility of FRACTALS was used to track the user operations 
in a comparative investigation of the user-friendliness of two human-interface 
devices, i.e. a mouse and the dials. 

TECHNICAL EVALUATION 

The technical evaluation of the Philips prototype was started without delay after 
its installation in the dedicated digital reading room. Immediately major 
shortcomings became clear concerning the image matrix size, the reliability, the 
performance and the facilities for evaluation. 
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The video scanner showed to have a bad image quality. Philips replaced it on 
request with a laser scanner. Only one single matrix size, I024xl024x8bits, was 
supported by the system. The ROC studies showed that the image quality 
obtained with this matrix was insufficient for primary diagnosis for various types 
of images. This implied that the clinical evaluation (see next section) could not 
be performed in the Radiology department and was thus restricted to the 
referring departments. The planned 1-screen workstation in the Internal 
Medicine ward was replaced by a 4-screen workstation in response to the urgent 
appeal of this department 

The system performance of the prototype appeared to be very limited. The 
network transfer rate was too low. The image management system did not 
support prefetching. 

There were no facilities for data security and data base integrity. The clinical 
evaluation study and the performance research were further hampered because 
of the lack of logging facilities on the prototype. 

Suggested improvements were, in contrast to earlier commitments, not realized 
because Philips discontinued the product line of the PACS prototype. 

CLINICAL EVALUATION 

The prototype PACS was implemented at one single ward (15 beds) of the 
Internal Medicine department. During the evaluation period, from October 1988 
until July 1989 (when the hospital moved to the new building) all old and new 
radiological images were supplied in digital form for all patients at that ward by 
means of the prototype PACS. Utilizing the HIS-PACS coupling it was possible 
to provide patient history and radiological reports together with the images. All 
radiographs were entered into the system through digitization with the laser 
scanner. This process appeared to be rather cumbersome and thus manpower 
consuming. 

The logistic process of diagnostic image distribution was analyzed for the film 
based system and using PACS. When films were used it appeared that about 
25% was not available in the central archive. Measurements showed that it takes 
about 1.5 days on average extra to retrieve these hard to trace images from their 
actual location. In the PACS situation this could be reduced to a few hours [7]. 

The initial skepticism of the Internal Medicine physicians turned into a growing 
enthusiasm about the functionality of the PACS prototype. They particularly 
appreciated the availability of the relevant patient data - images and related 
alphanumerical information - so soon after the acquisition. Also the certainty 
that the images were present, and accessible 24 hours per day contributed to the 
physicians' satisfaction. 

TECHNOLOGY ASSESSMENT 

Technology Assessment appeared to be a rather broad issue. Here we focussed 
on the financial aspects, i.e. costs and savings, of PACS implementation in a 
dutch hospital. 

The total costs of the prototype PACS have been calculated based on prices 
provided by Philips- AT&T. By extrapolating these costs to the full UUH it can 
be predicted that PACS is at present much more expensive than the 
conventional system. The outcomes of similar studies in the literature differ 
enormously. This clearly indicates the need for an international dialogue on the 
issue. 
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A PC software package has been developed to calculate first year costs of the 
PACS system versus the conventional film based system. The CAPACITY 
(Cost/critical Analysis of Picture Archiving and Communication Indicating its 
True Yield) package is based on three models: a model of the PACS, one of the 
hospital workload, and one of the price trend of each cost component. 
CAPACITY has been sent upon request to 10 research groups. The 
dissemination of CAPACITY has already proven to enhance the international 
dialogue. 

The results of CAPACITY based on the UUH figures predict that the break- 
even point will occur near the end of this century. A sensitivity analysis indicates 
that the costs of PACS largely depends on the required number of workstations 
and Computer Radiography systems, and the prices of these components [8]. 

The clinical evaluation also suggested that PACS will cause a reduction in 
length of patient stay. Whether such a reduction will lead to savings on the 
hospital budget depends on the country's reimbursement system. 



RECOMMENDATIONS FOR FUTURE RESEARCH 

The strategy for PACS in near future should not aim at hospital-wide PACS but 
at partial PACS. The subdivision can be realized along two dimensions. Firstly, 
instead of all departments only a few could implement PACS. Secondly, not all 
types of images are treated digitally, but either the digital modalities or the 
conventional projection roentgen images are included in the PACS. This implies 
that film has still largely to be used in parallel with digital images. 

From our PACS project we learned that the major technical requirements for 
PACS are: 

- sufficient image quality for primary diagnosis, 

- a user-interface, that allows an acceptable fast way of working, 

- an intelligent image management system, that includes prefetching, 
• a bi-directional coupling, 

- software logging facilities. 

Some hospitals and/or departements will benefit more from PACS than others. 
For example intensive care will have a great advantage from fast image delivery. 
A hospital with a spread-out floor plan will benefit more from PACS, because 
films do no longer have to be transported over the long distances. 

Research should include the following issues: 

- The effects on the working methods of the radiologist and on the 
organization of the Radiology department. 

- Modelling and simulation to support the hardware design and to guide the 
implementation. Also the development of software, such as image 
management strategies can be aided by simulation. 

- The diagnostic image quality of the new generation of PACS. 

- The user-interface of image workstations and the facilities for e.g. navigation 
through all images in a study. 

- Development of further integration of PACS with the hospital information 
system environment 
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- Structure and organization of the PACS data base system, including image 
management strategies. 

- Technology Assessment of PACS at various scales, e.g. per department, per 
hospital, nation wide. 

An extremely important tool for PACS research, in particular for clinical 
evaluation and performance studies, is a facility in the PACS system under study 
for the logging of all user operations and of the load of all components. 



REFERENCES 

1. BAZIS, UUH, Philips: Concluding report of the Dutch PACS project. 
Picture Archiving and Communication System. Phase I (1986-1989). March 
1990. Available upon request through BAZIS. 

2. Bakker AR, Kouwenberg JML, PACS/HIS an intimate relation needed. In: 
Proceedings AAMSI Congress 1989, 8th Annual National Congress, May 11- 
13, 1989, San Francisco, California, Edited by W Ed Hammond, pp. 220-224. 

3. Koens ML, Lodder, H, Bijl, K, de Valk, JPJ, Bakker, AR, HIS-PACS 
integration: experiences and future. In: Proceedings 9th Confenrence on 
Computer Applications in Radiology, June 1-4, 1988, Hilton Head Island, 
South Carolina, 1988. 

4. Stut jr. WJJ, de Valk, JPJ, ter Haar Romeny, BM, Bakker, AR, Modelling 
and Simulation within the Dutch PACS project, In: Proceedings Medical 
Imaging III Conference, January 1989, Newport Beach, SPIE 1093, pp. 423- 
428. 

5. Ottes, FP, Valk de JPJ, Kroon, HM, Barneveld Binkhuysen, FH, Seeley, GW, 
Winter LHL, Schultze Kool LJ, Diagnostic image quality evaluation chain: 
applications of the FEASIBLE package in practice. In: Proceedings Medical 
Imaging III Conference, January 1989, Newport Beach, SPIE 1093, pp. 2-9. 

6. Hofland PL, Ottes, FP, Vossepoel AM, Kroon, HM, Schultze Kool, U, 
Medical Imaging Workstation: A software environment. Journal of Medical 
Informatics, (in print). 

7. Haar Romeny ter BM, Wielen van der, JMM, Achterberg, AJ, Barneveld 
Binkhuysen, FH, Zuiderveld, KJ, Andriessen, JHTH, Bakker, AR, PACS 
efficiency: a detailed quantitive study on the distribution process of films in a 
clinical environment in the Utrecht University Hospital. In: Proceedings 
Medical Imaging III Conference, January 1989, Newport Beach, SPIE 1093, 
pp. 259-271. 

8. Poppel van, BM, Gennip van, EMSJ, Bakker, AR, Wilmink, JBM, First 
results of the software package CAPACITY for cost modelling of PACS. In: 
Proceedings Medical . Imaging IV Conference, February 1990, Newport 
Beach, SPIE 1234, in print 
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ABSTRACT 

Computed Radiology has been used routinely at the 
Mallinckrodt Institute for portable radiographs for over two years, 
although film has consistently been used as the medium for interpre- 
tation. We have recently set out to explore the feasibility of using 
electronic displays for both primary and secondary interpretation of 
portable chest radiographs. A prototype medical image presentation 
system supports a workstation-based user interface integrated with 
high-performance auxiliary displays. High-resolution displays allow 
for primary (using 2000 x 2500 pixels) and secondary (using 1000 x 
1000 pixels) interpretation. The user interface facilitates case selec- 
tion and access to patient information including radiology reports. 
Digital images are acquired directly from the CR unit and spooled to 
the review stations for interpretation. Experience with these proto- 
type systems will be reported. 
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Introduction 



Laser film scanners are commonly used to digitize plain 
radiographs for teleradiology and picture archival and 
communications systems (PACS). Laser scanners provide 
digital images of higher quality than those obtained by 
digitizing analog signals from a video camera. These 
scanners, however, may introduce new types of artifacts into 
the digital images. 

Many phantoms are available for testing various aspects 
of image quality for CT, MR and commercial video systems. 
We have recently described a quality control phantom for the 
digitization of radiographs/7/ which was designed to test 
several parameters of image quality, including resolution, 
low contrast discrimination, gray scale response, high 
frequency noise, and geometric distortion. 

Quality control protocols that focus on general 
parameters of image quality may not identify clinically 
important artifacts introduced by a specific imaging modality. 
In this paper we describe several artifacts which have been 
introduced into images by a commercial digital radiography 
system using a laser film scanner. A complete quality control 
protocol for digital radiology should incorporate an active 
search for these and other similar artifacts as an adjunct to 
the quantification of general parameters of image quality. 
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Materials and Method 



A copy of the Society of Motion Picture and Television 
Engineers (SMPTE) test pattern/2/ created on a 3M laser 
printer was obtained from the 3M Medical Imaging Systems 
Division (West Coldly, NJ). 

Multiple film phantoms were constructed by exposing 
commercially available line pair phantoms (Nuclear 
Associates, Carlo Place, NY) directly to film. These films 
were exposed at 50 kVp and 0.8 mass using a 0.6mm focal 
spot and a target to film distance of 56 inches. Physical 
phantoms were placed directly on the film cassette. No grid 
was used. 

Commercially available optical density step tablets were 
used (DuPont; Wilmington, DE). Other optical density step 
tablets were creating by exposing film with a 21 -step 
sensitometer (X-Rite; Grandville, Ml). The optical densities 
on these films were measured with a model 301 
densitometer (X-Rite). 

The various phantoms described above as well as 
multiple clinical images were digitized on a CommView 
digital radiography system (AT&T Bell Laboratories, West 
Long Branch, NJ and Philip Medical Systems, Shelton, CT). 
This system uses a model FD2000 laser film scanner 
(Dupont) operating at 2048 x 1684 pixel resolution with 12 
bits of data per pixel. During the period of this study, the 
CommView system was upgraded from the 1.0 software 
release with a results viewing station monitor to the 2.1 
software release with an enhanced graphics station monitor. 

Images were transferred to a personal computer 
workstation/3/ and truncated to the most significant 8 bits. 
They were visually inspected using the roaming zoom and 
contrast enhancement features of the ImagePro software 
package (Media Cybernetics; Silver Spring, MD). Additional 
quantitative analysis was performed with custom developed 
software to calculate mean and standard deviation values 
with the original 12 bit data for various regions of interest. 
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1 ) Streak artifacts 

Digitized phantom images may contain streak artifacts 
from the edges of very light areas surrounded by dark 
regions. This artifact was frequently seen with the SMPTE 
pattern (figure 1A). The degree of streaking was noted to 
vary with the position of the SMPTE pattern within the image 
matrix. Similar streak artifacts were not found to be 
noticeable in clinical images. 




A B 

Figure 1. Two SMPTE patterns digitized from a single sheet of x-ray film. 
4 corresponds to a pattern present on the upper half of the film. Streak 
artifacts are demonstrated at abrupt borders between bright and dark bars 
in the pattern. 

e was in the lower portion of the same film. Low contrast definition is 

2} Local variation in gray scale response 




Brightness and contrast may vary from point to point 
within a digitized image. For example, the two images 
displayed in figure 1 were obtained by simultaneously 
digitizing two identical SMPTE patterns from a single sheet 
of film. Nonetheless, there is a marked difference in both the 
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brightness and contrast of the two images. This artifact led to 
the discovery of a light leak in the laser scanner which was 
subsequently corrected. In addition, we have documented a 
consistent, though not visually perceptible, difference in the 
digital values assigned to various optical densities on the left 
and right sides of the image. 

3) Linear Artifacts 

Various types of linear artifacts may be introduced by 
either the laser scanner itself or by the software interface 
which transfers the image data into the PACS data base. 
Images obtained with the initial 1.0 software release of the 
CommView system demonstrated a linear artifact through 
the center of each image (figure 2). The appearance of this 
artifact varied with the contents of the phantom images. 




Figure 2. Linear artifacts introduced by the digitization of plain 
film phantoms. The appearance of the artifact varies with the 
phantom, 
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However, it was not appreciated on digitized clinical images. 
This particular artifact was not present after the system was 
upgraded to the 2.0 software release. The cause of this 
artifact was never determined, but was probably related to a 
software error in transferring data from the laser scanner. 

Other linear artifacts may be related to the actual 
hardware within the laser scanner. Figure 3 demonstrates a 
curvilinear dark line which was noted in the upper right 
corner of multiple phantom and clinical images. This artifact 
was corrected by replacing a scratched glass plate in the 
laser scanner. 




Figure 3. Note a curvilinear 
dark line introduced into the 
upper right quadrant of 
digitized images by the 
presence of a scratch on a 
glass plate in the digitizer. 



Figure 4 demonstrates two step tablet patterns which 
were digitized simultaneously from two sides of a single film. 
The digitized image contains multiple parallel linear artifacts 
through the darker portions of the step tablet which was 
present on the left side of the film. No such artifact is noted 
from the right edge of the film. Similar linear artifacts are 
present at the left border of clinical images. 
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Figure 4. Step tablet patterns following digitization. The images have been 
reoriented to facilitate comparison. 

A. Pattern on the right edge of the original x-ray film. The digitized image 
reveals no linear artifacts. 

B - Pattern on the left edge of the original film. Note the presence of linear 
artifacts in the darker portion of the step tablet. 




Figure 5. Moire patterns are introduced into digital images when the line pair 
frequency approaches the sampling frequency of the digitizing device. 
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4) Sampling frequency artifact 

This artifact is related to the resolving ability of the 
digitizing device, and will be present in all systems which 
digitize an analog image. Figure 5 demonstrates the Moire 
pattern which is seen when the line pair frequency of a 
phantom approaches the sampling frequency of the 
digitizing device. The appearance of the pattern varies with 
the orientation of the lines. 



5) High frequency noise 

High frequency noise in digital images may be visible as 
mottle. It is generally a greater problem in the darker 
regions of an image. Radiography systems that use a laser 
scanner to digitize images tend to have less of a problem 
with high frequency noise than the less expensive video 
camera based systems. Nonetheless, some high frequency 
noise will be introduced into the images by sampling error 
and "electronic noise"/4/. 

High frequency noise can be easily quantified by 
digitizing a step tablet/"*/ There should be no mottle present 
on the film image of the step tablet. Following digitization 
with the CommView system (figure 6A), the pattern is 
reproduced with no visible mottle. With a video camera 
system/3/, however, a significant amount of mottle is visible 
(figure 6B). High frequency noise was quantified by 
calculating the standard deviation of the digital pixel values 
within each step of the tablet. When this calculation was 
performed for the full 12 bit/pixel data of the CommView 2.1 
software release, a standard deviation of 4 to 5 digital units 
(for a gray scale of 0 to 1023) was measured in the lighter 
steps (0.2 - 0.4 density units). In darker areas (3.0 - 3.4 
density units) the standard deviation measured in the range 
of 60 to 100 digital units. 
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Figure 6, Digital images of photographic step tablets. 

A was tiicjiti/cci with a laser scanner system. No noise is visibly detectable 

B was digitized with a video camera frame grabber combination. A significant degree 

of mottle is present. 



Discussion 

A formal quality control protocol is essential to insure 
the diagnostic accuracy of any digital radiography system. 
Such a protocol should include regular inspection of 
digitized phantom images to quantify parameters of image 
quality. In addition, a small set of radiographs with specific 
clinical findings should be digitized and inspected on a 
regular basis. In this paper we have described several 
discrete classes of artifacts introduced by taser scanner 
digitization of radiographs. Familiarity with these specific 
artifacts may be helpful in the development of a quality 
assurance program for digital radiography. 
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CCD Mammography 
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Charge-coupled devices ( CCD ' s ) have 
exquisite sensitivity to light in the 
visible and near infrared portions of the 
spectrum. They are the main imaging 
receptor used by professional astronomy and 
are finding extensive use in many other 
disciplines. CCD's are a potentially ideal 
receptor for mammography and other areas of 
diagnostic radiology because of their wide 
dynamic range, real-time image acquisition 
and display, and low patient dose. They 
suffer from small size and relatively low 
spatial resolution. 

Experimental work has been performed 
with a LoRad mammography unit using a Trimax 
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8 screen coupled by a 50mm focal length 
f/2.0 lens to a Thomsom CSF TH7882 CCD. A 
mammographic phantom (CRIS Inc. model XI 1) 
served as a test object featuring both 
simulated high contrasts calcifications and 
low contrast nodules of different sizes. 

The early phantom results are 
encouraging (figures one and two) . At high 
magnifications the lens collects a 
sufficient amount of light for the image to 
be x-ray photon noise limited. Phantom 
images consistently have more dynamic range 
than comparable film screen images. Greatly 
increased contrast resolution can, in part, 
make up for more limited spatial resolution 
of the CCD system so that tiny 
calcifications are imaged rather well. CCD 
magnification views on the order of 3-4x 
compared to a standard film screen view show 
much more spatial detail (figures three & 
four) , but have a very limited field size of 
about 5cm by 5cm. With optimum electronics 



and optimum mammography screen-to-lens 
coupling, a CCD mammography system has the 
potential for reducing patient dose by a 
factor of ten. 

We will soon begin patient studies to 
investigate CCD mammography for "real-time" 
needle localizations , magnification views 
and specimen radiography. Large, high 
resolution CCDs now entering the market 
offer the possibility for CCD use in 
everyday mammography in the not too distant 
future . 
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FIG. 1. Film/ screen image of mammography phantom. 




FIG. 2, CCD camera image of mammography phantom. 
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FIG. 3. CCD camera image of microcalcif ications 
in mammography phantom. 
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FIG. 4. CCD camera image of low contrast nodules 
of mammography phantom. 



PACS Technical Workstation Design 



Evaluation of Input Devices for "Selection" Tasks 
in Diagnostic Workstations 

Kris Maloney, Kevin M. McNeill and John S. McNeill 
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Abstract 

A goal of the User Interface Study Group (UISG) at the Univer- 
sity of Arizona is to determine the requirements of a successful user inter- 
face for digital radiology. In pursuit of this goal we have conducted an 
experiment to determine I) what effect the input device has on the users 
performance, measured in time and accuracy, when using a digital work- 
station; and 2) if non-radiologists can participate in user interface studies 
that do not require diagnosis. Based on a model of the reading process, 
developed earlier by the UISG, a task was defined that required the same 
types of computer-human interaction as the reading process but did not 
require a diagnosis. Twenty observers, 4 radiologists and 16 non- 
radiologists, participated in this study. The results of this study indicate 
that the type of input device used has a significant impact on user perform- 
ance, and that in tasks that do not require diagnosis non-radiologists may 
perform similarly to radiologists. 



Introduction 

The use of the digital workstation, in 
all radiologic modalities, as a tool for pri- 
mary diagnosis is a critical factor deter- 
mining the success of a complete PAC 
system. Many of the issues that will de- 
termine acceptance of the workstation in- 
volve the physical characteristics of the 
digital workstation (i.e. spatial and con- 
trast resolution). These physical and per- 
ceptual requirements are being studied at 
the University of Arizona, as well as 
elsewhere, to determine what characteris- 
tics are necessary for a digital workstation 
to be used for radiologic reading. ]f2 



Also critical for the digital worksta- 
tion to be accepted as a tool for primary 
diagnosis is the transition of the radiolo- 
gist from the current film based reading 
methods to the new digital technology. 
This is a major obstacle that must be over- 
come to realize the complete benefit of a 
PACS. This obstacle is particularly diffi- 
cult to overcome because the current film 
based system works well for the radiolgist 
and has become ingrained in the diagnos- 
tic process. 3 

In order for the digital workstation to 
replace film based reading methods the 
transition must occur with as little disrup- 
tion as possible to the radiologists' learned 
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reading patterns. The radiologist must be 
able to read images without thinking 
about the computer. 4 Research in other 
areas has shown that a poorly designed in- 
terface can cause degraded user productiv- 
ity, user frustration, increased training 
costs, and the need to either redesign and 
reimplement the user interface or entirely 
replace the system. 5 In the field of radiol- 
ogy, these types of problems can be disas- 
trous. To effectively design a user inter- 
face that will be accepted and used by the 
radiologist it is necessary to understand the 
task which is to be performed by the radi- 
ologist. 6 To better understand the radiolo- 
gists' task we have formulated a model of 
the current film based reading methods. 7, 8 

Model of the Reading Process 
The reading model we selected to use 
in this experiment models the process for 
lightbox reading used for outpatient cases. 
The model was derived by analyzing video 
tape of six radiologists performing this 
type of reading. We chose this model be- 
cause it describes a task that is very cum- 
bersome and time consuming for the radi- 
ologist requiring significant direct 
manipulation of films. This model divides 
the reading process into three phases: 
Setup, Viewing/Diagnosis, and Wrap up. 
The following is an overview of the read- 
ing model. 

• Setup 

- * get next case. 

- * get demographic information. 

- dictate case demographics. 

• Viewing/Diagnosis 

- * get films from envelope and place 
on lightbox. 

- compare films. 

- dictate report. 



• Wrap-up. 

- * replace films in folder. 

- write diagnosis synopsis on folder. 

- * place folder on stack of com- 
pleted readings. 

Based on this film reading model we 
have further broken the reading process 
down into a series of elementary tasks and 
from this determined that most of the in- 
teraction by the radiologist with the film 
could be classified as "selection" tasks. 
Selection tasks are tasks where the user 
makes a selection from a set of alterna- 
tives (i.e. patient folder or desired film). 9 
Research in other areas has shown that 
one of the most important elements in the 
design of user interfaces is the choice of 
the devices and techniques by which the 
user performs elementary tasks. 10 

The tasks we determined to be com- 
pletely classifiable as a selection task are 
marked with asterisk (* above). The re- 
maining tasks consist of selection tasks 
but may contain other types of elementary 
tasks. Because of the significance of the 
selection task in this type of reading proc- 
ess, we designed an experiment to deter- 
mine if the input device used has an effect 
on the performance of the user while per- 
forming selection tasks at a digital work- 
station. 

The Experiment 

Goals 

There were two main goals in this ex- 
periment 

Hie first, as mentioned above, was to 
determine if the choice of input device had 
an effect on the overall performance of the 
user. 

The second goal of the experiment 
was to determine if, when given a specific 
non-diagnostic task, non-radiologists per- 
form similarly to radiologists. Conducting 
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research into the elementary components 
and techniques used in a specific interface 
design, although proven successful, is 
very time consuming requiring a large 
number of users to participate frequently 
in testing and experimentation. In the de- 
sign of a user interface for a digital work- 
station, the main requirement would be on 
the radiologists time. Adding non- 
radiologists to the user population would 
greatly relieve the burden on the radiolo- 
gists. 

Dependent Variable 

The data collected in this experiment 
was measured in terms of the dependent 
variable, performance. In this experiment, 
performance was measured in two ways; 
1) the time it took to complete the task 
and, 2) the number of errors that occurred 
when completing the task. 

Independent Variables 

There were three variables that we 
controlled and modified during the experi- 
ment. 

The first, and most obvious, variable 
we controlled was the input device used to 
perform the specified task. Hie subjects 
were randomly and evenly assigned either 
the mouse or the touch screen an the input 
device to be used throughout the experi- 
ment. 

The second variable we controlled 
during the experiment was complexity. 
The task was designed to include four 
complexity levels. The order in which the 
complexity levels were encountered and 
the amount of work required at each com- 
plexity level for each subject was constant 
throughout the experiment 

The third variable we controlled in 
this experiment was body position. To 
avoid biasing the results in favor of a par- 
ticular input device because of body posi- 



tioning we had each subject complete the 
required task twice; once in a sitting and 
one in a standing position. The subjects 
were randomly and evenly assigned to 
either stand first or sit first to avoid intro- 
ducing a training effect to the body posi- 
tioning results. 

The Digital Workstation 

This experiment was conducted on (he 
Arizona Viewing Console (AVC), a digital 
workstation here at the University of Ari- 
zona Department of Radiology. Input 
devices available on the workstation are 
three touch screen equipped monitors, one 
for command and the reniaining two for 
image data; function knobs used for win- 
dow, level, filter and continuous bilinear- 
interperolative zoom, a keyboard, trackball 
and a mouse. 

The Task 

We defined a task that involved the 
same type of activities as the film based 
reading model but that did not require di- 
agnosis. The following is a discussion of 
the basic design of the experiment 

Throughout the entire experiment there 
were two command menus displayed on 
the command monitor. One menu con- 
tained the commands (HORIZONTAL, 
VERTICAL, BOTH, and RESET) that 
would be used to manipulate image orien- 
tation during the experiment, the other 
menu contained a list of folders and the 
NEXT key. 

The basic task of the experiment was to 
manipulate the images in a series of 20 
folders to match a predefined pattern of 
how the images should appear. To do this, 
the images in a folder were displayed on 
the left hand image monitor and a goal card 
representing the same set of images, but at. 
different orientations, was shown to the 
subject The subject was required to select 
an image from the folder. For the purpose 
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of this discussion* the term "select" can be 
defined as; touch the named object, in the 
cases where the touch screen was the as- 
signed input device, or move the cursor to 
the object and depress the mouse button 
where the mouse was the assigned input 
device. Selecting the image from the 
folder on the left hand image monitor 
caused the image to be displayed on the 
right hand image monitor for manipula- 
tion. The subject then selected, from the 
menu on the command monitor, the type 
of manipulation, either HORIZONTAL 
(flip on the horizontal axis), VERTICAL 
(flip on the vertical axis), BOTH (flip on 
both the horizontal and vertical axis, and 
RESET (reset the orientation to the origi- 
nal orientation) required to make the im- 
age appear at the same orientation as the 
corresponding image on the goal card. 
The subject then selected the next image 
from the folder on the left hand image 
monitor and selected the appropriate type 
of image manipulation from the command 
monitor. When a new image was se- 
lected, the previously displayed image 
was updated in the folder display on the 
left hand monitor to include the manipula- 
tion. This process continued until the sub 
ject felt confident that all of the im- 
ages displayed on the left hand image 
monitor matched the images on the card. 
The subject then selected NEXT from the 
command menu and moved on to the next 
folder. 

The folders were ordered and dis- 
played to the subject in increasing levels 
of complexity. For the purpose of this ex- 
periment we have defined complexity to 
be the number of images in a folder. The 
first five folders contained one image, the 
next five folders contained two images the 
next five contained three images and the 
last five folders contained four images. 

To begin sixteen non-radiologists 
completed this task twice, once in a sitting 
position and once in a standing position to 



result in 32 sets of data. 

One person administered all of the ex- 
periments. 

Results - Non-Radiologists 

When measured at a 99% confidence 
level, the effect of the input device on user 
performance, for both time (Figure 1) and 
accuracy (Figure 2), for all complexity 
levels was statistically significant 
[F(l,124) = 43.58, P<.01] 




Figure 1. Average Times. 
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Figure 2. Average Errors 
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The body positioning had no statistical 
effect on user performance. 

We also noticed a statistically signifi- 
cant training effect at a 99% confidence 
level between the first and second time the 
subject completed the defined task 
[F(1,124)=7.15,P<.01]. 

The Radiologists 
We then added radiologists to the 
study. Analysis of data collected previ- 
ously, from the experiments with the non- 
radiologists, had shown that body position- 
ing did not effect the performance of the 
user. Because of this we decided it was 
unnecessary to have the radiologists per- 
form the same task twice with the same in- 
put device. Instead we had each radiolo- 
gist perform the specified task with both 
the mouse and the touch screen. The radi- 
ologists were assigned randomly and 
evenly to run either the mouse first and 
then the touch screen or the touch screen 
first and then the mouse. 

Because of the constraints placed on 
the radiologists time the single experi- 
menter who had conducted all of the non- 
radiologists experiments was not available 
to conduct the experiments for two of the 
radiologists. A second experimenter ad- 
ministered the experiments for these two 
radiologists. In analyzing the data we real- 
ized that this logistical problem had intro- 
duced a new variable to the experiment 
All of the non-radiologists and two of the 
radiologists received coaching from the 
experimenter. This coaching occurred 
when the subject became confused by 
either the task or the user interface of the 
workstation. The same type of instruction 
was given to all of the subjects if they en- 
countered a problem. The second person, 
conducting the experiment for two of the 
radiologists, did not provide the subjects 
with coaching of any type. 

Upon completion of the experiment 



each of the radiologists was asked if he or 
she felt the task accurately represented the 
"mechanics" of the film reading process. 
All of the radiologists felt that the task was 
a good representation of the film reading 
process. 

Results - Including Radiologists 
Figures 3 and 4 on the following page 
show the results of the experiments con- 
ducted with the four radiologists. Each ra- 
diologist is shown as a point of the graph 
with the radiologists who received coach- 
ing appearing as an open circle and the ra- 
diologists who did not recieve coaching 
appearing as a closed triangle. The verti- 
cal bars (to the left of the points represent- 
ing the radiologists) show the mean and 
standard deviation for the non-radiologists, 
for the first time they completed the task, 
with the mean represented as a black dia- 
mond and one standard deviation above 
and below the mean being shown as a hori- 
zontal bar. 

Discussion 

With the radiologists, as with the non- 
radiologists, the effect of the input device 
on user performance for all complexity 
levels and with and without coaching was 
statistically significant with a confidence 
level of 99%. Regardless of whether the 
mouse was used first or the touch screen 
was used first performance, measured in 
both time and accuracy, was significantly 
better when using the touch screen* 

The general trend of the radiologists 
receiving the same type of coaching as 
non-radiologists seems to indicate that 
non-radiologists may behave similarly to 
radiologists when performing specific non- 
diagnositic tasks. Additional radiologists 
will be tested to verify this trend. 

Although no statistical data can be in- 
ferred, coaching also had a major effect on 
the radiologists performance. Because of 
the signicant performance advantage no- 
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ticed for radiololgists receiving minimal coaching, questions arise regarding the need 
for intellegent and interactive help programs as part of the user interface for digital 
workstations. This area also requires additional study. 
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ABSTRACT 

Future PAC/IMAC systems will be acquiring, transmitting, 
storing, and displaying a variety of digital images. Film images 
will also be digitized and incorporated onto the system. The 
process of displaying digital images is one of the most crucial 
parts of such systems. These digital images will be displayed 
either on film or electronically. Much research is now being done 
to define the needs of these display systems. One of the ways 
researchers investigate the needs of PAC/IMAC display systems is to 
acquire a set of clinical images and digitize them. The resultant 
digital images are then manipulated in some way and then either 
written back to film or displayed on an electronic monitor. 
Reading performance for the original set and the duplicates are 
then compared using the receiver operating characteristic (ROC) 
experimental paradigm. For the results of these experiments to be 
interpreted properly the displaying of the digital image must be 
controlled such that they are matched as closely as possible to the 
original film images except for the variable (s) under study. This 
paper will describe some of the work being conducted at the 
University of Arizona Radiology Department for controlling CRTs and 
laser writers to conduct these type studies. 



INTRODUCTION 

A large amount of psychophysical/clinical research is now 
being conducted to investigate the needs of PAC/IMAC systems [1-4]. 
The questions being investigated have become more and more subtle 
over time as we learn more about what is needed. Thus, the need to 
control the experimental environment becomes even more pertinent. 
One of the ways researchers investigate the needs of PAC/IMAC 
display systems is to acquire a set of clinical images and digitize 
them. The resultant digital images are then manipulated in some 
way and then either written back to film or displayed on an 
electronic monitor. The original set and the duplicates are then 
compared using ROC analysis. For the results of these experiments 
to be interpreted properly the displaying of the digital image must 
be controlled such that they are matched as closely as possible to 
the original film images except for the variable under 
investigation . 

For example, if an experimenter wants to define whether the 
spatial resolution of a CRT display is sufficient for making 
diagnosis of a particular exam type, then the experimenter must 
make sure that the image on the CRT display matches the original in 
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every way except for the spatial resolution. If the other 
characteristics of the CRT are not controlled for, then the results 
of that study do not reflect only spatial resolution differences, 
but also all the other uncontrolled variables in the system. Thus, 
no specific conclusions about spatial resolution can be made. This 
paper (1) describes some of the techniques that we have developed 
to control CRT displays and laser writers and (2) presents results 
from studies that show the deleterious effects on perception when 
display characteristics are not controlled. 

DISPLAY TYPES 

CRT Displays 

When a digital image is displayed on a CRT there is usually 
not a one to one mapping of the digital values in memory to that 
which is seen by the observer at the surface of the screen. The 
CRT and the driver board combine to distort the values actually 
written to the screen. These two devices and the other electronics 
in the system contribute to the system greylevel/brightness 
transfer curve. Depending on the amount of change brought about by 
this transfer curve this distortion can have drastic effects on 
detection. A way to account for this is to characterize the 
transfer curve of the CRT system to be used in the research and 
then control its output using that information. 

We have developed a simple procedure for characterizing the 
greylevel/brightness transfer curve of a CRT system. First an 
SMPTE test pattern is written to the screen and the contrast 
brightness controls are manipulated until the 5% and the 95% 
difference squares are clearly visible. The brightness and 
contrast controls are then locked in place and a series of 
grey levels are written to the CRT screen and the light output is 
then measured in foot lamberts for each of the steps displayed. We 
use a simple silicon detector attached to a voltmeter with a 
digital display to take these measures. We attach the detector to 
the center of the screen, cover the screen to eliminate scatter due 
to ambient light and then take the measurements. We have used more 
elaborate controls and equipment to take these measures [5,6,7], 
but for characterizing the transfer curve of a CRT display system, 
we have found this simple technique worked as well as the other. 

Figures 1 and 2 are the transfer curves of two different 
display systems. The same CRT was used in both cases. The only 
difference was the driver board we used. One was hooked to our VAX 
8600 system. The other was in a PC based system. While both have 
non linear shapes, they are different. Putting the same 
information into the two systems results in different brightness 
outputs. We found that these differences have a significant effect 
on an observers detection capabilities. Figure 3 describes the 
results of a study we ran to investigate the effect of transfer 
curves on detection. We displayed the same set of images on the 
two different display systems characterized in Figures 1 and 2. 
The images had a noisy background and the signal, when it was 
present, was always in the center. There were five levels of 
signal. The images were shown to 6 observers one at time, in 
random order, and in a controlled experimental environment. The 
ROC experimental paradigm was used and the observer's task was to 
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tell us their certainty that a signal was present or not. The two 
curves in Figure 3 are the results of this study. As expected, in 
both cases as signal increases so does detection. However, the 
detection of the PC based CRT system is lower than that on the VAX. 
Efficiencies were calculated [(dH/dI)**2) ] comparing the human (dH) 
to an ideal observer (dl) and as can be seen the PC based system is 
35% [ (10-6.5)/10*100] lower than that of the VAX. 

Once you have determined the shape of the transfer curve it is 
a simple matter to make a look-up-table (LUT) to change the curve 
to any desired form. We photometrically linearized our displays 
and then ran a second study using the PC system to investigate the 
effect of linearization on detection. The same images and the same 
observers used in the previous study were used. The task and the 
experimental environment were the same. With photometric 
linearization there was a further decrease of 76.9% [(6.5- 
1. 5) /6. 5*100] in efficiency from the results of the previous PC 
study. This decrease in detection capabilities can be explained by 
the following: when these monitors were linearized, there was a 
reduction in slope of the transfer curve in the greylevel region in 
which the signal to be detected, resided. Thus, signal decreased 
for the observers. 

It should be noted that although the transfer curves depicted 
in Figures 1 and 2 are commonly encountered in a large range of CRT 
displays, other transfer curves are possible and they might lead to 
different effects. For example, if linearization has the effect of 
increasing the slope of the transfer curve in the range of 
grey levels in which the signal resided, then detection would most 
likely be increased. However for the range of displays that our 
transfer curves represent, photometric linearization is obviously 
not a good candidate for a general display transfer curve. 

These studies show how easy it is to affect the perception of 
observers and to distort they type of conclusions that are drawn 
from research using CRT displays. Next, we will discuss 
controlling laser written film to equate optical densities in the 
duplicate film to the original. 
Laser Written Film Displayed on Light Boxes 

An alternate way of displaying digitized images is to write 
them back to film and then display them using a light box. For 
example, suppose you wanted to investigate the effect of reduced 
spatial resolution on diagnosis. Affordable and available CRTs 
cannot display image matrix sizes much over 2000 X 2000 pixels 
with, at least, 8 bits of grey. This means that images having 
larger matrix sizes would have to either be displayed piecemeal or 
displayed by using a reduced matrix size. Neither method is very 
acceptable for this type of research, thus writing back to film is 
a popular method for investigations into spatial resolution needs. 
After the images have been digitized and the appropriate 
manipulations made to reduce resolution, but before writing them 
back to film, a crucial step must be taken. A LUT must be devised 
that will maintain the same optical densities in the duplicate 
images as those found in the original films. If this step is not 
taken before generating the images, then the results of such a 
study will be confounded due to this lack of experimental control. 

To avoid this problem we have developed a procedure to insure 



304 



that the optical densities of duplicate images will be virtually 
identical to those of the original. When we digitize the films to 
be used in our studies, we also digitize the National Bureau of 
Standards (NBS) x-ray step tablet. This tablet has 17 steps of 
optical density ranging from 0.20 to 4.15. For this work we only 
use the first 14. The 14th step has an optical density of 2.89. 
Next we use a computer program to find the average greylevel for 
each step in the digitized NBS tablet. We then calibrate our Kodak 
laser writer and film processor to stabilize the output and write 
out the digitized NBS step tablet. We also write out a digital 
step tablet that we create in the computer with a greylevel range 
dependent on the number of bits there are per pixel. For example, 
if the number of bits is 12, then we create one that ranges from 0 
to 4095 in steps of 32. We then process the two images and measure 
their optical densities. Having the digital and optical density 
values for both step tablets allows us to make a LUT that, when 
down loaded into the Kodak laser writer, adjusts the laser's output 
to equate the optical densities of the laser written NBS step 
tablet to those of the original. 

We did the above process for a set of images we are using in 
a spatial resolution study. Figure 4 shows the closest we can come 
to the original NBS step tablet using just the controls built into 
the Kodak writer. There is some correspondence up to around 2.20 
optical densities, but after that point the two diverge 
significantly. Figure 5 shows measurements of the laser written 
NBS step tablet with the same control settings as the previous one, 
but now with our LUT also incorporated into the system. There is 
virtually identical correspondence for all 14 steps. The last one 
is only off by 0.01 optical densities. Using the LUT we then wrote 
the images to be used in the resolution study. Figure 6 shows how 
closely the images we wrote using the LUT match the original. 
Sixteen locations were measured for one original image and the same 
16 locations were also measured in two duplicate images. One 
duplicate was written with and one was written without the LUT. As 
can be seen, the two curves are very similar to those we found 
using the NBS step tablet. Thus, we feel that we have produced 
images that vary only in terms of their spatial resolution and the 
results from our study will reflect only the manipulation of that 
parameter. As figure 6 demonstrates, without the LUT this would 
not have been true. 

DISCUSSION 

In the PAC/IMAC field researchers are asking subtler and 
subtler questions about system needs. Thus, the need for 
controlling all aspects of the experimental environment becomes 
even more important than in the past. We hope to have demonstrated 
in this paper, (1) that small differences in display 
characteristics can have large effects on perception, (2) that 
control of all aspects of the display system is necessary for valid 
results, and (3) that this control is also possible. Without 
characterization and control of the kind described in this paper, 
experimental results and conclusions will have very limited 
generalizability and usefulness. 
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ABSTRACT 

Siemens has developed and introduced a desktop workstation 
- the "LiteBox" - based on the Apple@ Macintosh@ computer. 
The focus of the development has been to build a powerful, 
easy-to-use, low cost imaging workstation. The LiteBox f s 
applications in medical imaging range from Radiation Therapy 
to Magnetic Resonance Imaging; and from Intensive Care Unit 
to referring physician's office. 

The development has raised several questions that a medical 
imaging equipment manufacturer has to consider and decide 
when introducing a personal computer-based product that 
impact the radiology end user and other medical staff. 
Some of these questions are: 

1. Is the "LiteBox 11 a system (hardware and software) 
offering or a software only product? 

2. What are the Medical Devices regulatory issues for 
a software only product? 

3. What are the service and maintenance considerations? 

This presentation will review the impact of these issues 
on the end user and the manufacturer. Other areas that 
will be reviewed are price/performance and medical-legal 
issues. 



308 



X Image Extensions to the X Window System Implications 
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INTRODUCTION 

This paper reports on the X Window System as an industry 
standard building block for creating the graphical user 
interface on bitmapped workstations and promoting open 
systems. It also reports on the proposed image extensions to 
the X Window System which enhance the architecture for 
building image capable information systems. The continuing 
decrease in the price/performance ratio in compute power, 
data storage and network bandwidth makes it technologically 
and financially feasible to incorporate image capabilities in 
all types of medical applications!!]. While the advances in 
medical imaging technologies have given physicians better 
diagnostic tools, the wide spread use of images in the 
healthcare environment will result in better patient care 
efficiency and increased physician productivity. A solid 
information system foundation is a critical predecessor to any 
image system. 

The X Window System provides an important standard and 
imparts a new approach to problem solving by allowing 
access to information resources throughout the hospital. This 
software architecture provides the ability to transparently 
access information and applications from anywhere on the 
network, across multiple operating systems and hardware 
platforms. As the use of images becomes more commonplace 
in all applications the need and use of standards will 
continue to provide benefits by decreasing the cost and time 



309 



310 



to develop applications. The use of these standards also 
insulates the applications from the underlying technology, 
protecting investments in software design and development. 

HISTORY OF THE X WINDOW SYSTEM 

The X Window System was originally developed under 
Project Athena at the Massachusetts Institute of Technology 
(MIT), jointly with Digital Equipment Corporation and with 
contributions from several other companies [2]. Widespread 
use of the X Window System in the Unix community was 
established with the release of Version 10. The X Window 
System continued to grow in popularity and to evolve with 
major enhancements in Version 11 and the current MIT 
release is X11R4. 

In January of 1988, the MIT X Consortium was established to 
continue the development of the X Window System and 
currently has over 70 members and affiliate institutions. It 
has as its major goal the promotion of cooperation within the 
computer industry for the creation of standard software 
interfaces at all layers in the X Window System environment. 

Two other significant activities which contribute to the wide 
spread use of the X Window System are also in process. They 
are the OSF and ANSI standard proposals related to the X 
Window System. 

The Open Software Foundation (OSF) has specified 
OSF/Motif as the application programming interface of an X 
Window System based environment. 

The ANSI standard proposal for the X Window System is in 
draft in the X3H3.6 Window Management Task Group. 

DESIGN GOALS OF THE X WINDOW SYSTEM 

One of the original goals of the X Window System ( or simply 
X) was to establish a portable device independent window 
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system for bit mapped displays. Device independence is 
achieved by separating the X Window System into two parts 
using the client/server model. The X server controls the 
hardware specific devices and implements instructions from 
the client. 

Not only was X designed to be hardware independent, it was 
also designed to be network transparent and network 
protocol independent. X clients communicate with X servers 
by sending packets of instructions using the X protocol. The X 
protocol is the metalanguage that describes how things will 
be done on the display and the interaction with the devices 
the X server controls. As long as the server has reliable 
communication to the client application, they may physically 
be across the network or reside on the same machine. By 
concise definitions of the semantics of the X protocol, it is 
made independent of the operating system, network 
transport technology, and the programming languages[3]. 

The X Window System provides the mechanism for drawing 
windows, but not the policy for user interface or using 
windows within applications. The window manager is 
responsible for imposing a policy and the look and feel of the 
user interface. 

Finally, the X Window System was designed to be extensible. 
This would allow certain classes of users to add features 
within the X framework[4]. 

X WINDOW SYSTEM ARCHITECTURE MODEL 

The core X Window System consists of 3 major components, 
the X server, Xlib and the Xtoolkit, as shown in figure 1. The 
X protocol separates the hardware dependent X server by 
means of Xlib and the Xtoolkit from the application. This 
arrangement means that applications can reside on different 
computers in a network and direct the output to any X server 
on which it has permission. Using the X protocol as the 
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window system metalanguage also means that applications 
do not have to be recompiled or relinked to execute on a new 
workstation/display. It is only the X server that is hardware 
dependent. 
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Figure 1. X Window System Client/Server Model 

X server is the code which runs on a workstation or X 
terminal. It provides a low level graphic interface to the 
display and user input capabilities, such as mouse and 
keyboard. 

Xlib is a library of X protocol access routines provided for the 
application programmer. By using Xlib routines the 
programmer has access to all X resources and results in 
highly transportable code from one operating system to 
another operating system. Xlib generates the appropriate X 
protocol messages for transmission to the X Server. The 
transmission may be by the network (DECnet or TCP/IP) or 
in the case when the X server and client are on the same 
workstation by shared memory or other interprocess 

communication. 
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The Xtoolkit provides additional routines to simplify the 
application access to Xlib. The two logical components of the 
Xtoolkit are called X intrinsics and X widgets. X intrinsics 
are well defined in the X Window Systems and are used to 
write higher level tools called widgets. Examples of widgets 
are pull-down menus, scroll bars, toggle buttons which 
implement features on the user interface. 

RELATIONSHIP OF DECWINDOWS TO X 

DECwindows is based on the X Window System and is fully 
compatible with X, but extends the architecture in a number 
of significant ways[7]. 

o Quality of implementation of the X components 

o Choice of programming languages interfaces 

o DECwindows Toolkit Widgets 

o Industry standard Xtoolkit interfaces 

o Extension libraries 

o Window and Session Managers 

o Terminal emulators 

o Personal productivity tools 

A comprehensive model of the DECwindows system is shown 
in figure 2. The client application can interface to Xlib, 
Xtoolkit intrinsics, Xtoolkit widgets, industry standard 
graphic libraries such as GKS, or to the X extension libraries. 

On the client side, the staircase layering of the application 
demonstrates the ability for applications to intermix calls to 
any of the client-side libraries. This important feature allows 
applications to take advantage of DECwindows toolkit if a 
high level of abstraction is desired, or the Xtoolkit if 
portability is desired. 



314 



— i 


i i 


Q] 
[ J 


Lent Applications 

■ — t 










Libraries 


Extension 
Libraries 

- Imaging 
-PEX 




j 

)ECtoolkit 




f 

X Intrinsics 




XUb 



I 

Transport Mechanism 



Transport Mechanism 


X Server Kernel 


Extension 
Libraries 



X Protocol 
X Server 



Figure 2 DECwindows Architecture Model 
X EXTENSIONS FOR IMAGING 



As with industry standard libraries, extensions can be made 
in the form of custom widgets- DECimage Application 
Services provides a set of custom widgets in the X 
environment which simplify the programming when images 
are manipulated[6]. Custom widgets typically are 
implemented using Xlib and X intrinsic calls[5]. 

The X architecture supports an extension facility so that 
functions can be added to the core routines. An extension 
consists of two parts, a hardware dependent extension to the 
X server and a client side library. Thus, X does not restrict 
hardware innovation for imaging or 3D graphic applications. 

Extension functions for image may include data 
compression/decompression, image scaling and rotation, 
image enhancement and color transformation. 
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The core X protocol provides a means for a client application 
to transmit and display images, however, it requires that 
most image manipulation take place on the client side of the 
application. Thus, applications can not take advantage of 
hardware acceleration which may be present in the display. 

X image extensions propose the following; 

o Image data extensions 

- including compressed images 
o Image manipulation and enhancement 

These move the device dependent processing functions from 
the client application into the server and allow the server to 
take advantage of hardware acceleration while still retaining 
device independence in the application. 

CONCLUSIONS 

The wide spread use of image applications in the healthcare 
environment will require the interoperability of a number of 
different computer systems and display devices. 

The X Window System in its current form provides software 
architecture which allows image applications to display 
images across a number of different computers, dsplays, and 
operating systems. 

The proposed X Image Extensions allow this same software 
architecture to take advantage of the hardware acceleration 
in these display devices and maintain portable image 
applications. 
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Introduction 

Since late 1984 the Massachusetts General Hospital (MGH) Department of 
Radiology has been actively developing PACS technology. This paper will 
discuss the overall system architecture of our most current development project, the 
Q-RSTAR (Radiology Storage Transfer Analysis and Reporting) system which has 
been under development for the last 18 months. We are currently installing 
portions of this technology in the department and intend to further deploy this 
technology throughout the hospital during 1990. 

The Q-RSTAR system is a direct outgrowth of our experience with its predecessor, 
the uRSTAR system, that has been used clinically for over two years at the MGH. 
The (iRSTAR system was intended to be a research system operating primarily to 
increase our understanding of the requirements for a full PACS. Q-RSTAR has 
been designed with the knowledge and experience gained from the jiRSTAR 
development and its clinical use. The main focus of Q-RSTAR is that of 
providing clinical utility to the department, the hospital, and affiliated institutions. 
Of the 16 Q-RSTAR workstations being installed in 1990 and early 1991, all but 
3 are for direct clinical use. 



Basic Q-RSTAR Functional Specifications 

The next few paragraphs will provide a cursory description of the basic functional 
specifications of Q-RSTAR at this stage of its development 

There are two classes of Q-RSTAR workstations: medium resolution, those based 
upon 1.2K x IK color or monochrome displays; and high resolution, those based 
upon 2K x 2.5K high frame rate monochrome displays. The workstations, based 
upon Apple Computer Macintosh, are designed to be multimodality, capable of 
manipulating CT, MRI, and various nuclear medicine images, as well as high 
resolution digitized radiographs. Animation and/or cini loops of selected image 
sets are provided for. The workstations will be 'multimedia' in the sense that they 
will provide for the spatial and temporal orchestration of image and graphics data as 
well as voice data. Image manipulations such as smooth zoom, flip, rotate, and 
window and center are provided for through the use of a local vector 
processor/pixel-mover. From our experience with jiRSTAR we have become 
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convinced of the need to have a capable, forgiving, yet simple user interface. In 
our effort towards reaching this goal we intend to deploy an interface guided by a 
number of expert systems to aid the user in 1991. 

The Q-RSTAR communication system is structured around a 'dual LAN* (local 
area network) concept: Ethernet for command & control information and a variety 
of high-bandwidth 'image blasters* for image transport within the MGH main 
campus and the MRI facility in Charlestown. This dual LAN concept yields an 
affordable communications system with an application performance for image 
transfer of over 10 megabyte/sec. 

Between the locations listed below we have in place standard telephone DS3 
circuits operating at 45 megabits/second as well as 1.5 megabits/second Tl lines 
carrying Ethernet to interconnect Q-RSTAR subsystems. These lines allow data 
interchange between all locations and provide application data transfer rates of over 
4 megabytes/second. The current locations are: 



• The MGH main campus in Boston 

• The PACS Development Center in Cambridge 

• The MRI Center in Charlestown 

• The Film Library in Somerville 

• The Spaulding Rehabilitation Hospital in Boston 

• An HMO in suburban Boston 



The image archive currently consists of two 1-Gigabyte (GB) optical drives with 
removable media (Digital Equipment Corp. RV20 drives), 6.8 GB magnetic disk 
arrays used as image caches (MGH designed), and 1.5 GB of magnetic disks used as 
image working disks. We are currently evaluating various juke-boxes to be 
integrated into the system. The jiRSTAR system supported an extensively 
catalogued off-line magnetic tape archive, and we are presently porting this 
software with enhancements to the newer Q-RSTAR system to support off-line 
optical media as well. The logical structure of the demographic database allows for 
the reception and generation of ACR/NEMA compatible data elements. 

Shared among the workstations is a multiplexed, 40 MIPS image processor 
capable of over 80 megaflops constructed from multiple Mercury Computer 
Systems Inc. array processors, model number MC-32-NU-8. 

The overall system is controlled by a Digital Equipment VAX 6220 currently with 
64 megabytes of main memory used solely for the RSTAR project. 
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Design Goals and Considerations 

In discussing the goals for the Q-RSTAR system, it is useful to view the goals 
from three perspectives: that of the user, of the department, and of the system 
designer. 

For the Q-RSTAR system to compete successfully with the existing film-based 
system with its over 60 film librarians, the Q-RSTAR workstation must: 1) 
present diagnostic quality images, 2) display these images in a fraction of a second 
on request, 3) be easy to use, and 4) be able to display any image known to the 
system at any workstation with equal ease and speed. 

From the department's perspective, if Q-RSTAR is to be viable it must be 
deployed throughout the department and the hospital. Thus the workstations must 
be affordable since there will be many of them. Therefore, the workstations must 
be built upon inexpensive platforms that also offer the necessary performance. 
Additionally, the overall system must be highly reliable . 

The designers of Q-RSTAR have taken the position that there will be no local 
storage at the workstations. Specifically, we wish to remove any geographical 
dependence at the workstation — any workstation has equal access to all images, 
via the high speed LAN, and to a large and high performance image cache. The Q- 
RSTAR system from its inception has been designed to be distributed, redundant, 
expandable, and modular. 
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Figure 1 : Shared Image Processor 



To meet the requirement of no local image storage at the workstation (that is, no 
large image storage disk; however, there will be an image cache consisting of high 
speed memory), we have designed what we call a Cluster-Support unit, (see Figure 
1). Each Cluster-Support unit contains a 3.4 GB magnetic disk array capable of 
transferring images at the approximate rate of 8 megabytes/second. By using the 
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information about active patients obtained from our radiology exam scheduling 
system (Digital Equipment Corp's. DecRad), we are able to place the necessary 
comparison and teaching files within the disk-image-cache as well as the images 
associated with the unread exams. Thus, upon a request from a workstation for a 
particular image, one that is likely to be found within the disk-image-cache, the 
Cluster-Support unit will be able to transfer the image data quickly to the optical 
image LAN. 

As shown in Figure 1, we use conventional Ethernet to transfer command & 
control information between workstations and the Cluster-Support unit. Since 
Ethernet is inadequate to send high resolution images in a timely fashion, we use 
an MGH designed 100 megabits/second optical LAN based upon Advanced Micro 
Devices Inc. TAXI components. 

Every user must have the ability to perform complex image processing in a timely 
fashion. A simple method of providing this functionality is to place at each 
workstation a high performance image and/or vector processor. The problem with 
this approach is that the workstation cost escalates with the inclusion of a device 
that will likely be used a very small percentage of the time. Instead, we have taken 
the approach of placing at the workstation minimal image processing hardware, 
sufficient for operations such as smooth zoom, flips, rotates, etc., and placing in 
the Cluster-Support module a high performance image processor that is shared or 
multiplexed between all the workstations (with its attending cost essentially 
amortized among all the workstations). For example, if a workstation wishes to 
perform a complex image processing task (e.g., an oblique reconstruction) the 
workstation will make a request via the command & control channel to the 
Cluster-Support unit It will transfer the images from the disk-image-cache to the 
shared image processor that will then perform the necessary calculations. Finally, 
the new images will be transferred via the high-speed optical LAN to the requesting 
workstation. Note that by using this architecture it is unnecessary to ever have to 
transfer image data back from the workstations to the Cluster-Support unit. Thus, 
the optical image LAN need only be capable of transmitting images in one 
direction — from the Cluster-Support unit to the various requesting workstations. 
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Figure 2: System Architecture 




As shown in Figure 2 above, we use a VAX computer for overall system control, 
as well as exam collection and formatting, managing the patient demographic 
database, managing the image archives, hosting various expert systems, and 
interfacing to the hospital information systems (HIS) and the radiology 
information system (RIS). The VAX Head-End is connected to one or more 
Cluster-Support units that in turn provide services for approximately 20 
workstations. Note, in the current configuration of Q-RSTAR, we are using the 
dual LAN concept as explained above; however, we plan to migrate to the FDDI 
(fiber distributed data interface) LAN standard as the cost of this technology 
decreases. 

Figure 3 outlines a Cluster-Support unit We base this subsystem on the Apple 
Computer Corp's. Macintosh computer for price and performance considerations. 
In particular, the NuBus standard supports sustained data transfers across the bus of 
37.S megabytes/second; we are thus able to transfer image data either from the 
image-disk-cache or image-cache-memory at high rates. The Cluster-Support unit 
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receives image data from its VAX master by a high speed parallel interface, a 
Digital Equipment DRB-32 on the VAX side. In some versions of the Q-RSTAR 
system the parallel interface's functionality is replaced by the existing Ethernet 
interface. 
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Figure 3 : Cluster Support 



Figure 4, shows a representative Q-RSTAR workstation. The Ethernet interface is 
used to convey command & control information between the workstation and the 
Head-End VAX and the Cluster-Support unit. High speed image data transfers 
from the Cluster-Support unit are received by the optical LAN interface and placed 
into the image cache & buffer. This buffer of between 4 and 16 megabytes holds 
the most recently used images and is used as a statistical cache. If the user 
performs repeated operations upon a small set of images, the needed image data is 
likely to be found in this cache negating the need to request service from the 
Cluster-Support unit. As mentioned earlier, each workstation contains a simple 
pixel-mover capable of very simple image processing operations. The primary 
function of the pixel-mover is to copy image data from the cache buffer to the 
display controller while at the same time performing the needed image transform 
such as flip, rotate, zoom, etc. The workstations support all Macintosh 
compatible display controllers, in particular the MGH designed 2K x 2.5K unit. 
Our experience to date indicates that the choice of Macintosh as the workstation 
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platform has been a good one. The wealth of software available, in particular user 
interface tools, has greatly enhanced our efforts. 
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Figure 4: Workstation Hardware 



Status of the Implementation 

At this time we have major portions of the system implementation and are 
currently deploying the technology in selected areas of the department. By year's 
end we expect to have the high-resolution workstations throughout the department 
as well as in selected areas outside of the main campus. In particular, we already 
have a high resolution digitizer (DuPont Corp. FD-2000) located at our Somerville 
film library that sends images to the main campus as needed. It is proving to be a 
most gratifying and exciting time for the MGH PACS development group and the 
department 
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PACS technology is at the point that a number of institutions are considering the 
purchase of radiology workstations. Yet no available design is appropriate for all 
clinical situations. This paper discusses the pros and cons of various computer 
human interaction or "navigation" design alternatives for a primary interpretation 
workstation. Other important non-navigation issues such as image quality, cost, 
and system integration are ignored. First, we detail a requirements taxonomy, fo- 
cusing on both the minimal features required for a typical clinical interpretation as 
well as more advanced characteristics. Second, we consider ease-of-learning with a 
discussion of the importance of mental models or metaphors. Third, we detail a 
number of critical issues effecting ease-of-use. Simple design evaluation tests are 
provided to help the reader understand the design issues as well as select the ap- 
propriate tool. 

REQUIREMENTS ANALYSIS 

Tools are only "good" or "bad" relative to a user and a task. Thus to evaluate a 
radiology workstation, we must determine who will use it to do what. This 
seems simple, but often tools are required to serve multiple users, each perform- 
ing a variety of tasks. We assume that the primary purpose of the workstation is 
viewing, by the radiologist, of images in the patient folder for the primary diag- 
nosis. Thus the workstation must first serve as a viable alternative to the four- 
over-four viewbox. Then (and only then) we can consider any "bells and whis- 
tles," which is to say more advanced features. It is all too easy to design a work- 
station focused on the bells and whistles, which ignores many of the more mun- 
dane, though essential features. 

Critical Function 

On our opinion, the following function is essential for a primary diagnosis radi- 
ology workstation. First, users must be able to locate specific patient folders. 
This requires integration with the Radiology Information System (RIS). Second, 
they must be able to rapidly locate specific studies or images within the patient 
folder. Third, radiologists must be able to display any image at full resolution. 
Fourth, they must be able to move sequentially through a multiple-slice study 



324 



325 



and roam around within a plain film larger than the display screen(s). Fifth, they 
must be able to side-by-side compare images from different studies, such as left 
and right mammograms, or this weeks and last week's CT chest study. Sixth, ra- 
diologists must be able to adjust the displayed image's intensity-window width 
and level, both by modifiable presets that vary with image type, and dynamically. 
While contrast-adjusting entire studies may be sufficient and is simpler, many ra- 
diologists may want to adjust individual images. 

"Bells and Whistles" 

The following features would be desirable for a workstation that successfully im- 
plements all the basics. First, tools for measuring distances across images. Such 
a tool would allow the radiologist to measure the length of a tumor or other 
anatomical object. Second, an electronic grease pen for highlighting critical 
anatomy for future readers. Third, zooming, either continues or bilinear. There 
may be situations in which zooming should be considered essential , but a sim- 
pler full resolution display with a pictorial index seems to provide sufficient spa- 
tial context for many modalities [Beard 1988] without the added hardware or com- 
puter-human-interaction complexity of the zoom operation. 

Fourth, 3D display of multiple-slice studies. Currently however, given the poorer 
quality of the images, 3D probably is of more use to surgeons trying to under- 
stand the spatial interrelationships of their patient's anatomy, than to the radiolo- 
gist generating an interpretation report. Fifth, contrast enhancement algorithms, 
which may be useful in many situations for improving feature detection. 

Functional Evaluation 

While one can simply read the marketing and technical documentation provided by 
the workstation vendor to determine function, it is best to sit down in front of 
the tool and actually try to use each one. It is important to realize that just be- 
cause a workstation has a particular feature, does not mean that an average radiol- 
ogist can actually use it. Thus the actual feature list of a workstation, as opposed 
to the advertised one, is highly dependent on the workstation's ease of learning 
and ease of use as detailed below. 

EASE-OF-LEARNING 

Automated office tools, like radiology workstations, can be threatening to a 
skilled professional: they eliminate the need for skills carefully developed over 
many years; they often force the user to submit to instruction by younger, less 
experienced individuals; and they can make the user feel like a child. These are 
problems with any new tool, but the harder a tool is to learn, the greater the af- 
fect. Thus ease-of-learning is critical to user acceptance of radiology worksta- 
tions. 
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To learn how to use a tool requires constructing a mental model [Rumelhart 
1981, Young 1981] of its operation. Mental models can be hard to acquire, so 
computer tools are often build to be analogous to some metaphor of physical real- 
ity such as a desk top [Macintosh 1988]. Mental models are used in radiology 
workstations to facilitate two tasks: locating studies within patient folders and de- 
termining their temporal and spatial/anatomical relationship, and as an image in- 
dex for locating images within studies and determining their temporal and anatom- 
ical position. We consider the pros and cons of several mental models currently 
used in radiology workstations. 
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Figure 1 

Desktop Metaphor with Manual Positioning 
of Individual Images 

Figure 1 shows a desktop metaphor used with some radiology workstations. 
With this mental model, individual images are selected from an image index, ei- 
ther a text index or a pictorial index consisting of miniatures of all images [Beard 
1988], and the individual images are separately moved to various positions on the 
display. This approach is simple to understand and works with both single and 
multiple studies [M c Neill 1987]. It is quite appropriate for viewing patient fold- 
ers containing a small number of large radiographs. However, the metaphor be- 
gins to breakdown when used with multiple-slice studies because of the consider- 
able number of hand-motions required and the fact that the display can become 
messy and confusing. While some workstations allow entire studies to be moved 
to multiple displays in one step, they still suffer from the problem of excess 
handmotions, because too many images must be individually moved for essential 
side-by-side comparison. 



Figure 2 shows another common metaphor in which each display has an image 
memory sufficient to contain part or all of an either study. The top three displays 
use the FilmStrip [Beard 1987] metaphor. It is most appropriate when viewing 
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multiple-slice studies. The bottom three use the metaphor of a viewport into an 
image display space. It is most appropriate when viewing large radiographs. 




□ □ □ 



Figure 2 

Individual Image Memories for Each Display 



However, only a limited amount of image memory is available to each screen, 
and often, there is not enough to display an entire study. Then the radiologist is 
forced to spend valuable time managing the memory resources of the computer. 
Further, since parts of a study my be spread among several displays, it becomes 
difficult for the radiologist to remember which slices of which study are located in 
which display. A great deal of time and effort may be spent simply moving stud- 
ies to the various displays and keeping track of their locations. 

The metaphors in figures 1 and 2 are appealing to system builders because of the 
parallels to the memory organization of the typical workstation. Because the 
user, not the tool, moves studies from disk and workstation memory to the vari- 
ous display memories, the developer can conceal slower data-path speeds and avoid 
developing complex memory management software. While this metaphor may 
reduce workstation cost, it is less than optimal for radiologists. 

We prefer to use the FilmPlane mental model (Figure 3), in which the images 
remain stationary relative to one another in the metaphor and the display or dis- 
plays are moved to the appropriate images. FilmPlane uses a metaphor of the 
lightbox, in which radiologists most often moves their eyes to the image, rather 
than the image to their eyes. The Viewports on the Navigation View in the cen- 
ter display can be manipulated with the mouse allowing any random image to be 
displayed on any display. Further, the individual Detail views on the left and 
right displays can be scrolled up, down, left, or right. A subject experiment con- 
ducted with a similar interface to a Hypertext system indicated that navigation 
views significantly improved information access [Beard 1990a]. 
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Figure 3 
FilmPlane Metaphor 

FilmPlane, as with most of the currently available radiology workstations, uses 
the direct manipulation interface style, Direct manipulation interfaces allow the 
user to "physically" manipulate metaphor objects on the screen, extending the 
mental model from comprehension to interaction. For an application involving 
visual objects such as medical images, the direct manipulation approach is almost 
essential. However, designing a viable direct manipulation interface is difficult, 
and there are many examples of systems that have a mouse, pull down menus and 
an impressive look, but are far too difficult to use. 

Ease-of-Learning Evaluation 

Users have learned the basics of a radiology workstation when they can locate and 
display a randomly selected image in the patient folder, sequentially move through 
a multiple slice study, side-by-side compare images from different studies, and ad- 
just image contrast both dynamically and with presets. We feel that with a well- 
designed workstation, a typical user should be able to confidently do these tasks 
after less than 15 minutes training. Ease-of-learning is best evaluated by bringing 
the tool in-house for a month or so and having the entire staff participate. But 
quick evaluations arc possible at trade shows as long as the users conduct the trial 
tasks with only little coaching by the observer or salesperson. 

EASE-OF-USE 

While ease of learning effects users' understanding of the system, ease-of-use ef- 
fects user's performance and frustration level. The following are several critical 
areas of ease-of-use. 

Response time: Response time requirements vary with the number of images 
being viewed and the average per-image viewing time. We suspect that response 
times for multiple-slice studies should be under one second for scrolling a screen 
with under a half second being desirable (this depends on the size of the screen). 
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The larger the image, the longer it takes to read it, so a longer response time for 
larger images should have less impact on total interpretation time. 

Errors: The interaction should be designed so that even beginning typical users 
make few errors e.g. selecting the wrong image, command, or mode. Most im- 
portantly, users should never have difficulty locating a study or image, or deter- 
mining from which study an image was taken. If one or more users continually 
repeat the same error, the fault lies with the workstation design, not with the 
users. 

Hand Motions: There should be an absolute minimum of hand motions re- 
quired for conducting a diagnosis, and one should almost never have to move in- 
dividual images. For example, FilmPlane2 requires only one keystroke per 
movement to side-by-side scroll through two CT chest studies. Many worksta- 
tions require over 10 keystrokes per movement to perform this same task. 

Display Area: Our work with an eyetracker [Beard 1989] indicates that there 
should be sufficient monitor space to show at least 12 CT images for CT inter- 
pretation. We estimate that even more would be desirable for radiographs [Pizer 
1989]. Primary interpretation workstations may require as many as four 2kx2k 
monitors. 

Intensity Windowing: The radiologist must be able to adjust the displayed in- 
tensity values of the images. A number of different approaches to dynamic inten- 
sity windowing have been tried including using mice or trackballs with horizontal 
motion controlling window width and vertical motion controlling level (or vice 
versa) as well as sliderbars and various buttons. Many radiologists have different 
interaction preferences depending on their experience and task, and dynamic win- 
dowing control is often discussed in the literature, but we are aware of no defini- 
tive study on this subject Dynamic intensity windowing requires a significant 
amount of time, so many radiology workstations offer preset buttons that auto- 
matically adjust an individual image or study to a some window width and level 
that is felt to best display particular anatomy. These preset values should vary 
with image modality with user modification possible. 

Ease-of-Use Evaluation 

The bottom line evaluation of any radiology workstation is whether the radiolo- 
gist staff can, under clinical conditions, generate acceptable interpretation reports 
in an acceptable amount of time. Thus, as with most tools, error rate and re- 
sponse time are the best objective measures of ease of use. But user confidence is 
also critical, for one does not want ill-at-ease radiologists who do not trust their 
tools. The workstation evaluator can develop a rough notion of ease of use by 
timing (and debriefing) a few radiologists reading multiple CT cases (or some 
other type of case with many images) after at least a half hour of training. But a 
most extensive evaluation should be conducted over a trial period of several weeks 
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to really learn whether that particular tool is appropriate for a particular institu- 
tion's particular users performing their particular tasks. 
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The forthcoming evolutton of the dlgl+al environment for radiology can be 
considered as three concurrent and Interacting evolutions. These are the 
evolution of component technologies, the evolution of system configurations and 
the evolution of professional and Institutional usage patterns. The value of 
the results of the technical effort In the first two areas are significant only 
In the framework of the future professional and Institutional utilization. 

A review of the current status of the many component technologies that are 
Involved In the digital environment Indicate that In most cases, one or more 
additional generations of Improvement are reaulred to satisfy the future 
requirements of the user and of the required systems configurations. The role 
of dlagnostlcal ly viable computed radiography for a multiplicity of 
examinations will probably be needed to Justify the existence of a general 
digital environment rather then the digital environment Justifying the 
utilization of computed radiography. For the systems configurations to achieve 
a general applicability within a radiology department, the system needs to 
progress enormously from the present status of selected modalities and limited 
Interfaces. 

The potential success and viability of this technical effort will depend upon 
the degree to which It can satisfy the evolving usage In the real world of 
radiology departments and the changing nature of the radiologist user. The 
usage Is expected to progress from providing a radiologist with prior 
correlative studies, often with reduced Information content, to providing the 
radiologist with images for original diagnosis for selected modalities, and 
then to providing the radiologist with Images of adequate quality In alt 
modalities for both diagnosis and demonstration. Concurrently the user can be 
expected to evolve from the research oriented radiologist-developer to the end 
point of acceptability by all radiologist users at progressively smaller 
Institutions. 

The acceptability of the future general digital environment to the 
Institutional user and the health care system will depend upon the potential 
benefits of such an environment, especially In the area of the ouallty and cost 
factors relating to the digital modalities and most especially to computed 
radiography. The convenience factors of the digital environment can become an 
additional benefit after a fairly broad level of applicability Is achieved. It 
Is expected that between 1.5 and 2.5 decades will be required to evolve 
component technology, system technology, quality benefits, cost and usage to 
support routine full utilization of the digital environment. 
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It Is believed that the successful evolution of the general digital environment 
will be highly dependent upon the effective utilization of computed 
radiography, using radiographic Imeges In a digital format such as sensitized 
plate technology. Computed radiography will probably be able to provide, 
Independent of Its Integration Into pictorial archiving and communication 
systems (PACS), Improvements In diagnostic Quality of certain examinations 
based upon Its wide dynamic range of radiation Input, Its support of new 
protocols of spatial and contrast post-processing, and Its support of computer 
aided diagnosis. 

If computed radiography can provide cost effective diagnostic quality benefits 
for a majority of radiographic examinations, then PACS, with considerable 
Improvements In component and systems technology and In system costs, can 
possibly provide Improved image file availability and other convenience 
benefits, provided that radiologists can be provided with and can accommodate 
feasible facilities for rapid video Interpretation. 

The following outline provides an evaluation of the expected evolution of 
component technologies, evolution of system configurations, and evolution of 
usage and user. 



1. EVOLUTION OF COMPONENT TECHNOLOGIES 

1.1 CENTRAL PROCESSING UNIT 

FUNCTIONS Data acquisition management 

Data storage and retrieval management 
Data distribution management 
PIS, HIS Integration 

Management of communications with other networks or nodes 

FACTORS Archive memory transfers 

On-line, Interim, cache memory transfers 
Processing speed 
Multiple processors 

Data compression & restoration (required to reduce archive 

storage, some transmission times) 
Selected Images archived vs all Images generated protocols 

(required to reduce archive storage) 
Redundancy, reliability, and fault sequences 
Confidentiality, security 
Work station case acaulsltlon time 
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K? ON-LINE MEMORY STORAGE 

TYPES Dally activity memory - Centralized (may be In CPU) 

Dally activity memory - Distributed 
Interim memory - Images generated and not archived 
Cache memories - Queues for transfers or 
Queues after transfers 



FACTORS Da 1 1 y act Ivlty memor I es 

New and retrieved data 

Centralized or distributed by modality & subspeclal fty 
Usually not compressed 
Cache memories - Queues for transfers between memories, 
especially between archive and distributed dally 
memories, or queues of new and retrieved data waiting 
Interpretation 



I .3 ARCHIVAL STORAGE 

TYPES Optical disk drive units 

Optical disk Jukebox 
Optical taoe 

Terabyte level units required 



1 .4 DATA ACQUISITION 



MODALITIES 



FACTORS 



CT, MR, US, NM, DSA, Dig Fluoro, 

Computed Radiography (CP) - Portable Films 

Computed Radiography (CP) - General 



TYPE Raw data 

Full reconstruction 
Processed data 



As stored 
As viewed 



DATA 512x512 pixels x 8-14 bits-most digital Imaging modalities 

QUALITY IKxlK pixels x 12 bits - probably Inadequate for 

radiography 

2Kx2K pixels x 14 bits - probable adequate for radiography 
4Kx4K pixels x 16 bits - probably not required for 

radiography 

INTEGRATION Local dedicated Interpretation first, then Images to PACS 
Single modality central dedicated processing I 
Interpretation first, then Images to PACS 
Transfer to PACS before Interpretation (required) 
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1>5 DATA PRESENTATION WORK STATION 

PUPPOSF Display prior, correlative studies 

Selected modality Interpretation, correlative studies 
Multi-modal Ity Interpretation, correlative studies 

(required level) 
Image display for demonstration 

FUNCTIONS 

DISPLAYS Multiple screens - t, 2, 4, more 

Quality 1Kx1K pixels, 1/60 sec - now available 

2Kx2K pixels, tow brightness- I Im I ted availability 
2Kx2K pixels, high brightness, required for 
radiography 

PROCESSING Slow manipulation (present status) 

Medium speed processing (limited availability) 
Very high speed processing (required level) 
Rapid case acquisition time 

ADVANCED VERY RAPID IMAGE MANAGEMENT & NAVIGATION 



1,6 TECHNICAL WORK STATIONS - INPUT/OUTPUT 

TYPES Film digitizers - Medium resolution, available 
- High resolution, available 
Hard copy printers, available 
High resolution laser printers, available 



1.7 COMPUTED RADIOGRAPHY (CR) - SENSITIZED PLATE TECHNOLOGY 

CHARACTERISTICS 

Viable technology Independent of PACS Integration 
Wide dynamic range of radiation Input 

Supports new spatial and contrast post processing protocols 
Supports computer aided diagnosis 
Inherently digital date 

FORMAT Cassettes 
Magazines 

Integrated components (chest, radiographic tables) 

FACTORS Format 2Kx2K pixels x 2 bytes 
8 MBytes/ Image 

IMAGE Local dedicated Image processing, film output-available 

PROOFS SING Local dedicated Image processing, digital output-available 
Central dedicated Image process I no-corn I ng 
PACS Integrated Image process t ng-requ I red 

APPLICATION Based on quality factors 

Based on convenience factors 
Typical sequence by exam type 

Portable, OR, chest, abdomen, GI/GU, head, spine, 

angiography, extremities, mammography 
Typical sequence by patient type 

ICU patient, pediatric, outpatient, ER satellite, 

Inpatient 
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1 .8 REMOTE NETWORKS 

FUNCTIONS Multl location practices 

Te I erad lolopy Inputs - for d 1 apnos 1 s 
Te I erad lology conferencing 

Remote demonstration - emergency rooms, operating rooms 
Remote demonstration - offices, clinics, nursing units, 
conference rooms 



1.9 DATA TRANSMISSION 

TOPOLOGY Ring, star, I lne 

Integrated, mufti node, distributed hubs and clusters 
Subnetworks, multiplexers 



TYPES 



Modems 
Wire 

T-1 Line 
Ethernet 



Fiber cable 
Fiber (FOOD 
ANSI high speed channel 
Very high t Ime-bandw I dth 
products required 



COMPRESSION Destructive - Prl to more 

Non-destructive - 2:1 typical for digital Images 
3:1 typical for radiography 



EVOLUTION OF SYSTEMS CONFIGURATIONS 



2.1 VARIOUS CONFIGURATIONS 



SI ngle d Igltal modal fty 

Multiple digital Imaging modality PACS 

Computed radiography local area network (requires considerable 
devel opment) 

Partial radiology department PACS (reoulres great component and 

systems development, cost Improvements) 
Ful I radiology department PACS (enormous requirements, an eventual 

goa 1 ) 
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2.2 QUALIFYING FACTORS 

Digital modal It fes 

Transfer of adequate acquired digital data 
Feasibility of video Interpretation 
Speed of video Interpretation 
System functional Ity 
Cost 

Computed radiography 

Demonstrated diagnostic quality benefits of selected 

computed radiography over film-based radiography 
Feasibility of video Interpretation 
Speed of video Interpretation 
Component adequacies and capacities 
System functionality 
Cost 

Full Integrated multimodal Ity systems (radiography and digital 
Imaging) 

Demonstrated diagnostic quality benefits of majority of 

radiographic exams by CR or digital techniques 
Feasibility of video Interpretation 
Speed of video Interpretation 
Component adequacies and capacities 
System functionality 
Cost 



3. EVOLUTION OF USAGE AND USER 

3.1 SYSTEMS USAGE 

FUNCTION Provide radiologist with prior, correlative studies 
Provide single digital modality Images for diagnosis 
Provide several digital modality Images for diagnosis 
Provide radiographic Images for diagnosis 
Provide several modality Images Including radiography for 

diagnosis and demonstration 
Provide all modality Images for diagnosis and demonstration 
Allow very rapid Image management A navigation 

USER Research oriented radiologist-developer 
Selected multiple radlologlsts-users 
Acceptance of selecting reduced number of Images for 
archive 

All radiologist users - selected large Institutions 

Consensus status 
All radiologists users - most large Institutions 
All radiologists users - progressively smaller Institutions 

TIME Will probably require 1.5 to 2.5 decades to evolve component 

technology, system technology, duality benefits, 
cost and usage to support full utilization 
Partial Implementation at selected Institutions In the 
Interim 
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3.2 POTENTIAL BENEFITS - THE PURPOSE OF IT ALL 

Qua I Ity factors 

Digital modalities - availability of undegraded digital 
data for video Interpretation 

Computed radiography 

Wide dynamic range of radiation Input 

(For demanding body parts and broader technfaue variations) 
(Radiation statistics must still be satisfied for adequate 
Images) 

Supports new protocols of spatial end contrast Image processing 

(For enhanced diagnosis for some examinations) 
Supports computed aided diagnosis for some examinations 

Convenience factors 

Provides alternative to film-based film file limitations 

Rapid availability of previous studies for prompt reading and 

prompt demonstration 
Availability of same Images for multiple exams and In multiple 
locations 

Peductlon of non-available films, lost films 

Convenient availability of Images for consultation, conferences 

Provides greater flexibility In the physical location of the 
radiologist for Interpretation of studies 

(Supports modality & subspecialty specializations) 
(Supports multiple locations of service) 
(Supports centralized teaching environment) 
(Does not alleviate need for radiologist In multiple 
locations to perform studies end contact patients) 

Provides greater flexibility In the physical location of the 
referring physician for the demonstration of radiologic 
studies 

Interrelationship of benefits 

Basic PACS desirability and feasibility will derive from the 
Inherent digital nature of a growing number of modalities 
Including radiography 

The diagnostic quality factors of computed radiography for a 
multiplicity of examinations will probably need to Justify 
PACS before PACS can Justify the remainder of digital 
radiographic examinations for convenience factors. 

But, with adeauate patience, It Is probably much more than a house 
of cards built upon a foundation of smoke and mirrors. 



3.3 COST VS. BENEFIT 



Must be satisfied 



A Quantitative Projection of Digital Environment Activity 
for Radiology Departments 

Arthur F. Naylor 

A.F. Naylor Company, Radiologic Consultants, Cleveland, OH 



A detailed quantitative model has been developed for the digital environment 
activity within a hospital radiology department utilizing digital technology 
(FACS) to varying extent for different radiologic modalities and 
subspecialties. The Information presented can either be used as a step by step 
guide for preparing a similar model for a particular Institution or as 
Indicative of the numeric magnitude of the required systems and the sensitivity 
to a number of factors and protocols. 

The model Includes provision for the number of yearly examinations for each of 
the radiologic modalities and specialties, provision for the number of Images 
that are generated for each type of examination and provision for the digital 
content of each type of Image. The distribution of yearly examinations Is 
typical of a teaching hospital and the quantity Is normalized to 100,000 total 
examinations per year. 

From this data the yearly number of examinations, Images and size of required 
digital archival memory can be evaluated for each of several levels of assumed 
PACS technology utilization. Five such levels of digital Interpretation, 
archiving, retrieval and presentation are considered! 1) digital Imaging 
modalities, 2) digital Imaging modalities plus portable x-ray studies, 3) the 
above plus 25% of the diagnostic x-ray fluoroscopic and radiographic studies, 
4) the above plus 50% of the diagnostic x-ray fluoroscopic and radiographic 
studies, and 5) utilizing the digital environment for all radiologic studies, 
Including angiography and mammography. 

By using this Information with the results of a prior study that provides an 
extensive model of patient visit patterns and archive growth, retrieval and 
retention patterns, the analysis Is extended to Indicate the total dally 
digital activity Including both generated and retrieved data. 

The key factors assessed are the magnitude of the required long term storage 
depending upon the modalities involved and the library age for the digital 
storage system; and the amount of memory required for a typical days operation 
Including both generated and retrieved data. This dally activity generates the 
requirements for data transfer rates and for compression and decompression 
processing times. 

The data Indicates the sensitivity of the digital systems requirements to 
variations In examination mix and professional protocols between Institutions. 
The study suggests the strong desirability of the selection, at the time of 
interpretation, of certain Images of each study for primary long term retention 
with the remaining Images retained either for only a short period of time or 
retained only with high compression ratio Irreversible techniques. The effects 
of library age and retention period are also Indicated, 
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The following tables can be used to Indicate the required magnitude of the 
digital archiving and retrieval activity within a hospital radiology department 
with a number of varying conditions and protocols, or can be used as a guide In 
preparing an Institution specific similar model, Table 1 establishes the 
digital Image format for each of the radiologic subspecialties and modalities 
and Indicates a typical protocol for the average number of Images per 
examination for each category. Table 2 shows a distribution of yearly 
examinations by subspecialty and model Ity that would be typical of a teaching 
hospital. This distribution Is given per 100,000 examinations per year and 
this table and the others that follow can be scaled to any desired size of 
departmental activity. Based on these parameters Table 3 Indicates the 
quantities of digital data that would be generated yearly both with and without 
data compression. A non-destructive data compression of 3.1 to 1 was assumed 
for the radiographic work and a non-destructive compression of 2.1 to 1 was 
assumed for the digital modalities. The table Indicates the amount of digital 
data generated for five different degrees of Incorporation of digital archiving 
and retrieval Into the department operation. These levels are: digital 
Imaging modalities only (ultrasound, nuclear medicine, computed tomography, 
magnetic resonance and DSA), digital Imaging modalities plus portable 
radiographic studies, the previous categories plus one-quarter of and plus one- 
half of the diagnostic x-ray radiographic and fluoroscopic studies and a last 
level with all studies handled digitally. 

Table 4 Indicates the amount of digital data that would be generated on a 
typical fairly busy day for each of the levels of digital activity. Table 5 Is 
based on prior studies of patient visit patterns and an archive growth model. 
It Indicates the average number of previous examinations that might be expected 
to be retrieved for a patient who has had one or more previous visits based on 
different ages of the digital Image library. When one Is dealing with patient 
visit patterns and a dynamic file model It Is extremely Important to 
distinguish carefully between patient examinations, patient visits, persons per 
year and the number of persons or files In the library. Based upon the dally 
activity and the dynamic file model It Is possible to calculate the quantity of 
generated and retrieved digital data on a typical day for each level of 
operation. This data Is shown on Table 6. It should be noted that these 
quantities correspond to a yearly activity of 100,000 examinations and would be 
the data for Just one days operation. 

Table 7 shows a summary of the total archived data and the new and retrieved 
digital data per day for a library age of seven years, a retention period of 
five years and the protocols of all Images archived and all flies retrieved. 
The Indicated number of archival units required Is based on an assumed capacity 
of 250 gigabytes for each archival unit. The dally active memory requirement, 
which might either be maintained In a central processing location or In several 
dispersed locations. Is based on an Individual active memory capacity of eight 
gigabytes. Table 8 Indicates how the requirements for archival storage and 
dally memory for new and retrieved data grow with library age. 
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In order to evaluate the sensitivity of the digital requirements to changes In 
protocol , alternate models have been formed with the assumption that, at the 
time of Initial Interpretation, a certain number of the Images generated for 
each exam would be designated by the radiologist as primary Images. The 
expected number of the Images that would be designated as primary for each 
subspecialty or modality Is Indicated on Table 9, together with the resulting 
calculation of primary Image megabytes per examination. Table 10 shows t ne 
corresponding quantities of archival Information that would be generated yearly 
with and without compression for e8ch level of operation. It should be noted 
that with this protocol an additional short term erasable memory would probably 
be advisable for temporary retention of the non-selected Images. The size of 
this temporary memory Is based on a six day retention. Based upon this 
alternative protocol of archiving primary Images only, the archival 
requirements and active memory requirements can be calculated. These 
requirements are shown on Table 11 for different levels of digital operation 
and are shown on Table 12 for different years of library age. 

An additional protocol that could be considered would be to utilize mixed 
amounts of compression for the archiving of Images. In this case It Is assumed 
that the primary Images would be stored with non-destructive compression as 
established earlier and the non-primary Images would be stored with a non- 
reversible compression of approximately 20 to 1. It should be noted again that 
the selection of primary Images needs to be made at the time of original 
Interpretation. Table 13 shows the resulting requirements for archival storage 
and for dally active storage of new and retrieved data for different levels of 
digital operation. Table 14 shows the requirements for different years of 
I Ibrary age. 

The comparison of the long term archival requirements per 100,000 yearly 
examinations and for a library age of seven years with the different protocols 
Is provided In Table 15. Similarly Table 16 provides a comparison of the dally 
active memory requirements for the different protocols. It can be seen that a 
substantial reduction In archival storage requirements and dally active memory 
requirements can be achieved through the use of the alternative protocols. 



Table 1. DIGITAL IMAGE FORMAT 









M-BYTES 


GEN. 


GEN. 


EXAM 


MATRIX 


DENSITY 


PER 


IMAGES 


M-BYTES 


TYPE 


PIXELS 


BYTES 


IMAGE 


/EXAM 


/EXAM 


DX-FL 


2K2 


2 


8,000 


8 


64.0 


DX-RAD 


2K2 


2 


8.000 


2.1 


16.8 


DX-MAM 


2K2 


2 


8.000 


6 


48.0 


DX-ANG 


2K2 


2 


8.000 


40 


320.0 


OX-PORT 


2K2 


2 


8.000 


1 


8.0 


DX-W/O PORT (AVE) 




4.2 


33. P 


DX TOTAL (AVE) 






3.3 


26.0 


DSA 


• 5K2 


2 


0.524 


12 


6.3 


US 


.5K2 


1 


0.262 


40 


10.5 


NM 


.25K2 


2 


0.131 


6 


0.8 


CT-HD 


.5K2 


2 


0.524 


20 


10.5 


CT-BD 


.5K2 


2 


0.524 


30 


15.7 


MR-HD 


.5K2 


2 


0.524 


25 


13.1 


MR-BD 


.5K2 


2 


0.524 


35 


18.3 


DIGITAL IMAGING (AVE) 




27.0 


10.9 



Table 2. TYPICAL YEARLY EXAMINATION 
DISTRIBUTION PER 100,000 EXAMINATIONS 



EXAM YEARLY 

TYPE EXAMS 

DX-FL 5,000 

DX-RAD 45,000 

DX-MAM 3,000 

DX-ANG 2,000 

DX-PORT 24,000 

DX-W/O PORT 55,000 

DX TOTAL 79,000 

DSA (NOT PRIMARY EXAM) 

US 6,000 

NM 2,750 

CT-HD 4,000 

CT-BD 4,000 

MR-HD 3,000 

MR-BD 1,250 

DIGITAL IMAGING 21,000 

SUMMARY 

DIGITAL IMAGING 21,000 

DIG IM + PORT 45,000 

+25? DX (FL * PAD) 57,500 

+50?! DX (FL 4 RAD) 70,000 

ALL STUDIES DIGITAL 100,000 
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Table 3. SUWARY OF DIGITAL DATA GENERATED YEARLY 
WITH AND WITHOUT DATA COMPRESSION 



COMPP. 



EXAM 




IMAGES 


ARCHIVE 


1Yr ARCH 


TYPE 


EXAMS 


1000's 


G-FYTES 


GPy 


DIGITAL IMAGING 


21,000 


581.3 


235.2 


111.5 


DIG IM + PORT 


45,000 


605.3 


427.2 


173.5 


+25* DX (FL A PAD) 


57,500 


638.9 


696.2 


260.2 


+50* DX <FL & PAD) 


70,000 


672.5 


965.2 


347.0 


ALL STUDIES DIGITAL 


100,000 


837. e 


2,287.2 


773.5 



Table 4. SUWARY OF DIGITAL DATA 
GENERATED ON A TYPICAL DAY 



EXAM 




GEN. 


GEN. 


TYPE 


EXAMS 


IMAGES 


M-BYTES 


DIGITAL IMAGING 


107.5 


2,906 


1,176 


DIG IM + PORT 


227.5 


3,026 


2,136 


+25* DX (FL * RAD) 


290.0 


3,194 


3,481 


+50* DX <FL 4 RAD) 


352.5 


3,363 


4,826 


ALL STUDIES DIGITAL 


502.5 


4,189 


11,436 
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Table 5. TYPICAL IMAGE 
LIBRARY GROWTH TABLE 



AVE. HIGH 
FILF FX/RET EX/RFT 
AGE YEARS RATIO RATIO 



1 


1.000 


0.875 


1 .917 


2 


2.000 


1.500 


4.833 


3 


3.000 


2.021 


5.806 


4 


4.000 


2.455 


6.616 


5 


5.000 


2.817 


7.291 


6 


5.625 


3.079 


7.854 


7 


6.125 


3.284 


8.262 


8 


6.521 


3.445 


8.581 


9 


6.830 


3.570 


8.831 


to 


7.067 


3.666 


9.025 


n 


7.889 


4.000 


9.694 



FOR PATIENTS WITH A PPEVIOUS VISIT 
PETENT I ON PROTOCOL: 5 YEARS AFTER OLA 



Tablo 6. SUMMARY OF DIGITAL DATA 
GENFRATED AMD RETRIEVED ON A TYPICAL DAY 



GENFPATED 

EXAM GEN. RETRIEVED I RETRIEVED 

TYPE G-BYTES M-BYTES G-BYTES 



DIGITAL IMAGING 


1.18 


3.14 


4.32 


DIG IM + PORT 


2.14 


5.64 


7.78 


+25$ DX (FL & RAD) 


3.48 


9.18 


12.66 


+50$ DX (FL & RAD) 


4.83 


12.70 


17.53 


ALL STUDIES DIGITAL 


11.44 


30.04 


41.48 



YEARLY ACTIVITY 100,000 EXAMS 

LIBRARY AGE 7 YEARS ALL IMAGES ARCHIVED 

RETENTION PERIOD 5 YEARS ALL FILES RETRIEVED 
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Table 7. ARCHIVE AND RETRIEVAL SUWARY - ALL IMAGES ARCHIVED 



DAILY 

COMPP. NEW/ APCH. MEMORY 
APCH. RETRIEVED UNITS UNITS 
GBy GBy/DAY §250 GB §8 GB 

SUMMARY 



DIGITAL IMAGING 


683.1 


4.32 


2.73 0.54 


DIG IM + PORT 


1,062.4 


7.78 


4.25 0.97 


+25* DX (FL & RAD) 


1,593.9 


12.66 


6.38 1,58 


+50? DX <FL & PAD) 


2,125.4 


17.53 


8.50 2.19 


ALL STUDIES DIGITAL 


4,737.4 


41.48 


18.95 5.19 



YEARLY ACTIVITY 100,000 EXAMS 

LIBRARY AGE 7 YEARS ALL IMAGES APCHIVED 

RETENTION PERIOD 5 YEARS ALL FILES RETRIEVED 



Table 8. LIBRARY GROWTH SUMMARY - ALL IMAGES ARCHIVED 





COMPR. 


NEW/ 


ARCH. 


MEMORY 


PPARY 


APCH. 


PFTRIEVED 


UNITS 


UNITS 


AGE 


GBy 


GBy/DAY 


0250 GB 


§8 GB 


1 


773.5 


19.44 


3.09 


2.43 


2 


1,546.9 


25.16 


6.19 


3.14 


3 


2,320.4 


29.93 


9.28 


3.74 


4 


3,093.8 


33.90 


12.38 


4.24 


5 


3,867.3 


37.21 


15.47 


4.65 


6 


4,350.7 


39.61 


17.40 


4.95 


7 


4,737.4 


41.48 


18.95 


5.19 


10 


5,465.7 


44.98 


21.86 


5.62 


n 


6,101.7 


48.03 


24.41 


6.00 



YEARLY ACTIVITY 100,000 EXAMS 
ALL STUDIES DIGITAL 
ALL IMAGES ARCHIVED 
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Table 9. SELECTED IMAGES FOR PRIMARY ARCHIVE 





GEM. 


PRIM. 


GEN. 


PRIM. 


EXAM 


1 MAGES 


1 MAGES 


M-PYTES 


M-PYTES 


TVDC 

1 Yrc 


/caAM 


/EXAM 


/EXAM 


/EXAM 


DX-FL 


8 


5 


64.0 


40.0 


DX-RAD 


2.1 


2.1 


16. e 


16.8 


PX-MAM 


6 


6 


48.0 


48.0 


DX-ANG 


40 


5 


320.0 


40.0 


DX-PORT 


1 


0.4 


8.0 


3.2 


DX-W/O PORT (AVE) 


4.23 


21.45 


33.8 


21.5 


DX TOTAL (AVE) 


3.25 


15.91 


26.0 


15.9 



DSA 


12 


6 


6.3 


3.1 


US 


40 


12 


10.5 


3.1 


NM 


6 


6 


0.8 


0.8 


CT-HD 


20 


12 


10.5 


6.3 


CT-BD 


30 


12 


15.7 


6.3 


MR-HD 


25 


12 


13.1 


6.3 


MR-BD 


35 


12 


18.3 


6.3 


DIGITAL 


IMAGING (AVE) 27.03 


4.74 


10.9 


4.7 



Table 10. ARCHIVAL STORAGE OF PRIMARY IMAGES 

COMPR. 6-DAY 
EXAM ARCHIVE 1Yr ARCH TEMP. MEM 

TYPE EXAMS G-BYTES GBy "B" MB 



SUMMARY 

DIGITAL IMAGING 

DIG IM + PORT 
+25$ DX (FL I RAD) 
+50? DX (FL I RAD) 

ALL STUDIES DIGITAL 



21,000 99.6 

45,000 176.4 

57,500 415.4 

70,000 654.4 

100,000 1,356.4 



47.2 4,068 

72.0 9,828 

149.1 10,728 

226.2 11,628 
452.6 30,228 



TEMPORARY SIX DAY STORAGE OF NON-PRIMARY IMAGES 
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Table 11. ARCHIVE * PETRI EVAL SUMMARY - PRIMARY IMAGES ARCHIVED 



DAILY 6-DAY 

COMPR. NEW/ APCH. MFMORY MEM B 

ARCH, RETRIEVED UNITS UNITS UNITS 

GBy GBy/DAY 6250 GB §6 GP §12 GB 

SUMMARY 



DIGITAL IMAGING 


289.1 


2.51 


1.16 


0.31 


0.34 


DIG IM + PORT 


440.8 


5.33 


1.76 


0.67 


0.82 


+25* DX (FL 4 PAD) 


913,1 


9,82 


3.65 


1.23 


0.89 


+50? DX (FL I PAO) 


1,385.3 


14.30 


5.54 


1.79 


0.97 


ALL STUDIES DIGITAL 


2,772.3 


30.12 


11.09 


3.76 


2.52 



YEARLY ACTIVITY 100,000 EXAMS 

LIBRARY AGE 7 YEARS PFDUCED IM/EX ARCHIVED 

RETENTION PERIOD 5 YEARS ALL FILES RFTRIEVED 



Table 12. LIBRARY GROWTH SUMMARY - PRIMARY IMAGES ARCHIVED 



COMPR. ARCH. MEMORY MEMORY 

LIBRARY ARCH. RETRIEVED UNITS UNITS B UNITS 
AGE GPy GBy/DAY §250 GB 08 GB §12 GB 



1 


452.6 


17.05 


1.81 


2.13 


2.52 


2 


905.2 


20.44 


3,62 


2.55 


2,52 


3 


1,357.9 


23.27 


5.43 


2.91 


2.52 


4 


1,810.5 


25.62 


7.74 


3.20 


2.52 


5 


2,263.1 


27.58 


9,05 


3.45 


2.52 


6 


2,546.0 


29.01 


10.18 


3.63 


2.52 


7 


2,772.3 


30.12 


11.09 


3.76 


2.52 


10 


3,198.5 


32.19 


12.79 


4.02 


2.52 


n 


3,570.7 


34.00 


14.28 


4.25 


2.52 



YEARLY ACTIVITY 100,000 EXAMS 
ALL STUDIES DIGITAL 
REDUCED IM/EX ARCHIVES 
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Table 13- ARCHIVE A RETRIEVAL SUMMARY - MIXED COMPRESSION 

DAILY 

COMPP, NEW/ ARCH. MEMORY 

ARCH. RETRIEVED UNITS UNITS 

GBy GBy/DAY 0250 GP 06 GB 

SUMMARY 

DIGITAL IMAGING 327.5 2.51 1.31 0.31 

DIG IM + PORT 514.5 5.33 2.06 0.67 

+25? OX {EL & RAD) 995.9 9.82 3. 96 1.23 

+50f DX <FL & RAD) 1 ,477.3 14.30 5.91 1.79 

ALL STUDIES DIGITAL 3,054.2 30.12 12.22 3.76 



YEARLY ACTIVITY 100,000 EXAMS MIXED COMPRESSION FOR ARCHIVE 

L I BPARY AGE 7 YEARS (20:1 FOR NON-PRIMARY IMAGES) 

RETENTION PERIOD 5 YEARS ALL FILES RETRIEVED 



Table 14, LIBRARY GPOtfTH SUWARY 
MIXED COMPRESSION 





COMFR. 


NEW/ 


ARCH. 


MEMOPY 


IPRARY 


ARCH, 


RETRIEVED 


UNITS 


UNITS 


ACE 


GPy 


GBy 


250 GB 


8 GB 


I 


498.7 


17.05 


1 .99 


7.13 


2 


997.3 


20.44 


3.99 


2.55 


3 


1,496.0 


23.27 


5.98 


2.91 


-1 


1,994.6 


75.62 


7.98 


3.20 


5 


2,493,3 


27.58 


9.97 


3.45 


6 


2,P04.9 


29.01 


1 1 .22 


3.63 


7 


3,054.2 


30.12 


12.22 


3.76 


10 


3,523.7 


32,19 


14.09 


4.07 


n 


3,933.8 


34.00 


15.74 


4.25 



YEARLY ACTIVITY 100,000 EXAMS 

ALL STUDIES DIGITAL 

MIXED COMPRESSION FOR ARCHIVE 



Table 15. COMPARISON OF ARCHIVE MEMORY REQUIREMENTS 



ALL IMAGES ARCHIVED 
UNCOMPRESSED DATA 

ALL IMAGES ARCHIVED 
COMPRESSED DATA 

ALL IMAGES ARCHIVED 
MIXED COMPRESSION 

PRIMARY IMAGES ARCHIVED 

ALSO PFOUIPES TEMP. MEMORY 



ARCHIVE ARCHIVE 

GBy UNITS 

14,009 56.0 

4,737 19.0 

3,054 12.2 

2,772 11.1 



YEARLY ACTIVITY 100,000 EXAMS 

LIBRARY AGE 7 YEARS ALL STUDIES DIGITAL 

RETENTION PERIOD 5 YEARS 250 GB /MEMORY UNIT 



Table 16. COMPARISON OF DAILY ACTIVE MFMORY REQUIREMENTS 

NEW AND DAILY 

RETRIEVED MEMORY 

GBy/DAY UN I TS 

ALL IMAGES ARCHIVED 41.48 5.2 

ALL IMAGES ARCHIVED 30.12 3.6 
MIXED COMPRESSION 
PRIMARY IMAGES RETRIEVED 

PRIMARY IMAGES ARCHIVED 30.12 3.6 



YEARLY ACTIVITY 100,000 EXAMS ALL STUDIES DIGITAL 

LIBRARY AGE 7 YEARS B GB/M EMORY UNIT 

RETENTION PERIOD 5 YEARS 



Operational Issues Confronting PACS: 
What Needs to be Done 

Steven C. Horii, Seong Ki Man, Betty Levine, et a/ 
Department of Radiology, Georgetown University Hospital, 
Washington, DC * 



ABSTRACT 

Technology has advanced to the point at which picture archiving and 
communications systems (PACS) have become feasible. While it is neither 
inexpensive nor simple to install such a system in a hospital, a number of 
institutions have done so. 

In our experience, the use of the hardware once installed showed a distinct 
time lag. Though there were those who began to use the system immediately upon 
receiving their workstations, these tended to be the non-radiology users. When 
questioned, the radiologists had few complaints about the images. Rather, it was 
marked differences between the way the PACS worked as opposed to conventional 
film reading which formed the nucleus of the dissatisfaction with the system. 

Though most of these issues have been addressed with the result that use 
among radiologists has increased tremendously, the process of identifying the core 
operational differences and determining how they could be eliminated or ameliorated 
was very educational In some cases, work-arounds were implemented to bypass 
stumbling blocks. In other instances, we had to wait until system software was 
updated before necessary features were available. 

Occasionally, the system was used enthusiastically even though such use 
meant changes in operating routine. What were the incentives in these instances ? 
Most commonly, the incentives were a time savings and reduction in frustration 
when compared to retrieving films from the file room. 

This paper will summarize our experience in moving our PACS from a 
technical success to a clinically useful tool, and will describe the operational features 
implemented and those still necessary. 
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INTRODUCTION 

During the past three years, we have taken the Georgetown PACS from a 
large scale prototype to an operational, clinically useful system. There is still a 
large amount of work remaining before the system can be said to have completely 
replaced conventional methods of operating the imaging side of a radiology 
department We have learned a great deal from these three years, and will be 
applying what we know to the further growth of the system. 

Most of the work which needs to be accomplished in order to take the system 
further is not in adding hardware. The development of more tailored software is far 
more important 

This paper will examine the system from the point of view of its major 
functional blocks, and will provide information on what we think is necessary for 
growth in each of those blocks. A schematic diagram of the Georgetown PACS is 
shown in Figure 1. 



IMAGE ACQUISITION 

The Georgetown PACS (or Image Management and Communications System 
- IMACS in our terminology) faced the problem of how to acquire image data from 
diverse equipment. The Department has two GE 9800 Quick CT machines, a 
Siemens l.ST Magnetom MR scanner, four ultrasound machines (Acuson, two 
ATL, and Diasonics),and DuPont laser film digitizers. In order to acquire images 
without having to develop interfaces for each device (the ACR-NEMA Standard 
interface was not yet available when the system was being installed) the method of 
video digitization was used. This has worked remarkably well despite its 
limitations. 1 For ultrasound, the quality of the images has been judged to be good 
enough for primary reading from the workstations. Though the spatial resolution of 
the CT and MR images is excellent, the contrast resolution is limited to 8 bits 
which is insufficient for primary interpretation (unless multiply windowed images 
are acquired). 

Our first recommendation in image acquisition is that video 
digitization is probably sufficient only for ultrasound imaging* 
The use of digital interfaces for MR and CT is highly 
recommended. 

In acquiring images, we rapidly discovered that there is an important human 
element, the technologist, who must be consulted when a system will affect his or 
her work patterns. In earlier versions of the IMACS, the technologist had to type in 
a full set of patient demographics, though they discovered that they could leave out 
everything except the name and record number. 




Figure 1: Georgetown University Hospital PACS Configuration 



353 



When we replaced the CT CRT multiformat camera with a laser film writer, 
we started receiving complaints about the speed of acquisition. The response time 
of the acquisition module (AM - contains the video digitizer and interface to the 
database manager) was longer than that of the laser camera, whereas it had been 
shorter than that of the CRT imager. At this point,then, sensitivity to IMACS 
performance was altered by a non-IMACS device. With the use of in-service 
training and developing close cooperation with the technical supervisors, we were 
able to overcome many of the technologist objections to the extra steps required to 
input images into the IMACS. Major advances in our favor were the improvement 
in AM response time with new software releases and implementation of an interface 
to the RIS (more on this later). Both had the effect of reducing technologist waiting 
time and workload. 

The second lesson we learned is that the impact of new 
operations on technical staff must be carefully studied* If workload 
cannot be decreased, then the benefit and necessity of performing 
the additional steps must be made clear to the staff. 

Most recently, because we now have workstations in multiple locations 
outside of radiology, we realized that getting image data from a machine is not 
enough. What is one to do with the data? When workstations were few in number, 
or infrequently used, the simple answer was to store it in a database and let it be 
accessed as needed, or broadcast it to all workstations. Real operations require that 
some selectivity be incorporated into the process. The question of how the image 
data is to be routed is one which affects both acquisition, since the examination type 
tends to be device or modality specific, and the database. 

We learned from this problem (which is not yet completely 
solved) that the imaging equipment may have to provide some 
examination identification so that the image data can be linked to 
other information in the database* 

Film digitization has been our main method for input of radiographic images. 
The question of whether or not the method can yield images good enough for 
primary diagnosis is still open. We digitize all films from the Pediatric Intensive 
Care Unit (PICU) and Intensive Care Nursery QCN) as these two locations have 
workstations. We discovered that this process is labor-intensive, and that reliability 
of the hardware needed to be very high. On the plus side, we discovered that the 
technologists were willing to undertake this extra work partly because they 
discovered that they were not being interrupted as often during the day by physicians 
looking for films. 

On the basis of our experience, if film digitization must be 
used (as it will have to be for input of old or outside studies) then 
some form of bulk-loading digitizer, which can read bar codes or 
other identifiers on the films, would greatly simplify this 
operation. A computed radiography (CR) system with a direct link 
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to the PACS is probably a better solution for new imaging (at least 
for portable and pediatric films)* 



IMAGE DISPLAY 

For the sectional imaging modalities, such as CT, MR, and ultrasound, 
which generate images typically in the 512 x 512 pixel size range with 8 to 12 bits 
of gray scale, the displays of the present workstations are adequate. For CR images, 
which can be up to 2048 x 2560 pixels, the utility of a monitor which can display 
one-quarter of that data (1024 x 1280 pixels) has been questioned. 

Display resolution issues aside, we have found that the human factors 
character of the workstations is a much more powerful influence on how well the 
workstations are accepted. If the image quality is excellent, but the process of 
bringing studies up on the monitors is laborious or confusing, it will take a very 
large incentive to make anyone use it An example of this is with our own 
radiology information system (RIS) report approval process. The first pass at 
supporting electronic signature at a terminal was so cumbersome (if all one had to 
do was sign reports, the process was reasonable, but the review process also included 
editing the reports) that a large fraction of the staff (even those who are computer 
literate) refused to use the system. After the complaints were addressed, the 
acceptance rate improved tremendously. 

There is a continuing debate as to whether the graphical user interface (GUI) 
is the best one to use for workstations. The majority of technical (not necessarily 
medical) workstations do use some form of GUI, which would lend support to the 
idea. We have some experience with a GUI for a workstation which we designed in- 
house.2 This user interface was preferred over the one currently employed at our 
radiology diagnostic stations. The one advantage of the user interface at the 
workstation is that it is consistent with that used for acquisition, database access, 
and on the smaller PC-based systems. 

We think that whatever user interface is employed at a 
workstation, its features should be tested in the field with attempts 
at doing real work* We have found that some features work well 
for demonstration purposes, but turn out to be cumbersome when it 
comes to production work. 

A recurring problem we have noted is the shift in expectation when using a 
workstation. Timing studies we have done 3t4 have shown that the time to bring 
up a case on the workstation is not very different from the time to do so from films 
in a jacket. The time is a bit longer for the workstation compared to a preloaded 
alternator when doing sequential accesses. For random retrieval, the workstation can 
beat the alternator. The user perception, though, is that the performance of the 
workstation is inferior to the film and alternator. We think that a part of this 
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perception has to do with the expectation which a user brings to the system. The 
mental model is one of "it's a computer, so it's fast". Even to those of us who 
understand factors such as disk access speed and bus transfer rate, staring at a blank 
screen seems to feel like an interminable wait A part of the good user interface idea 
is to provide feedback to the user when a process will leave the display idle for some 
time. 

Our belief is that further increases in display speed will be 
useful and appreciated, but that the demand for ever more speed will 
continue. Feedback to the user when delays are anticipated should 
be an integral part of the user interface. 

Radiologists each have slightly different ways in which they do their reading. 
This is not so much inter-observer variability in interpretation, but subtle 
differences such things as the order in which small tasks are done or films are 
arranged. Residents quickly learn these preferences, and provide for them during the 
reading sessions. 

There has been considerable discussion of the potential for 
such customization of workstations, but we have not seen such 
ideas implemented. Setting preferences based on the user log in 
would probably help improve user acceptance of the workstation. 

The necessity of design of the reading environment appropriate to 
workstations has been discussed by the authors before 5 We have continued to see 
that very simple changes, such as increased work surface space and improved access 
to telephones, can make a difference in how comfortable it is to use a workstation. 
Task lighting, environmental noise, traffic patterns, and temperature and humidity 
control are all factors which can make a work place comfortable or impossible to 
use. Architects and engineers have studied such elements of work room design for 
years, and have more recently included designs which accommodate CRT-based 
equipment. 

We think that a careful consideration of the environment into 
which workstations will be placed is essential, and that competent 
architectural advice should be obtained. The PACS vendors should 
be able to provide some assistance in this regard. 



COMMUNICATIONS 

The Georgetown IMACS incorporates a number of communications links. 
For connection between the database and workstations, and acquisition modules and 
the database, high-speed fiber optic links are used. These provide the high 
performance needed for these applications. We support teleradiology applications 
with two communications systems. One application, recently terminated because of 
the conclusion of a contractual agreement, used a T-l service, a connection available 
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through telephone companies. This supports 1.544 megabit per second data transfer 
which is much faster than the highest speeds (19200 bits per second) possible with 
current modems on voice-grade telephone lines. Voice-grade lines arc also used, 
however, for dial-up access to the IMACS database from PC-based workstations, our 
other tclcradiology application. For the research environment, and for supporting 
the PC-based workstations which are within the Hospital, Ethernet is used. It 
connects to the IMACS via a gateway, and currently supports retrieval of images 
from, but not input to, the IMACS. Video links to the acquisition devices from the 
imaging modalities are accomplished over analog coaxial cable, and operate at video 
rates. Twisted-pair lines are used for serial communication to the various text 
terminals supporting acquisition, display, and system control functions. 

This seems to be a complicated set of communications methods to support, 
but this diversity has allowed us to tailor the method used to the requirement When 
we have pulled fiber optic cable, we also pull an Ethernet cable so that we will have 
a choice of which connection to use. This has proved valuable in allowing us to 
exchange workstations based on evolving user needs without having to wait for a 
new cable run to be pulled. 

The lesson we have learned is that this aspect of our PACS 
was properly planned, and we advocate that other systems also be 
able to support a variety of communications options. 



ARCHIVING AND THE DATABASE 

Georgetown uses a star configuration for its PACS. The archive and database 
computer form the center of the star. The short term archive consists of magnetic 
disks with a total capacity of 8 gigabytes. For reliability, this capacity is normally 
halved, so that the data is written to two disks (or "mirrored"). We find that this 
capacity supports approximately 10 to 14 days of image data at present levels of 
acquisition. We currently schedule movement of the data to the archive after five 
days, but this is a temporary measure until mirroring is re-enabled. The long 
term archive is an optical disk jukebox with a total capacity of 178 gigabytes. We 
can store more than one year's data at present image generation rales. 

All of these values will be severely impacted by the input of CR images, or 
the expanded use of film digitization. This is because these images arc so much 
larger in data volume than those from CT, MR, or ultrasound. We have already 
seen that even keeping the images in the magnetic files for a week is not long 
enough in many cases. For inpatients, this probably reflects the bias towards the 
very ill patients in the intensive care units, which have been the major clinical areas 
supported by the system. For outpatients, this seems to be due to the length of 
time of the "workup" cycle. If a patient is not very ill, the time between one 
examination and the subsequent one can be more than a week. The University of 
Kansas data 6 would suggest ten days as a good value for storage time on short term 
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media. We would concur, and add that perhaps retaining data on line for the average 
length of stay plus some offset would be another target. 

A thorough examination of the projected data volumes and an 
analysis of the patterns of use are important in configuring the 
storage component of a PACS. Erring on the side of too much 
capacity will certainly result in a lower frustration level among 
users, though it will cost more to provide this capacity. 

The star design has a flaw in that it is subject to catastrophic failure if a 
single point, the central computer, fails. To avoid this sort of problem, the IMACS 
has a number of features which allow for "graceful degradation" with failures. The 
mirroring of the image and text data has proven highly reliable. We have had hard 
disk failures, but no image loss as a result. The database itself is provided with an 
uninterruptible power supply (UPS) to allow it to ride out short power outages, or 
provide time to do an orderly shutdown. We found out through a power failure that 
the system works, but we made one planning error. There were no terminals 
connected to the UPS, so though the database went on running, there was no way to 
shut it down properly. We solved the problem by unplugging an unused peripheral 
and plugging in a terminal so that a proper system shut down could be done. This 
procedure ties up all the computer "loose ends", so that restarting is very much 
simplified, and no data is losL 

Each workstation has local storage as do the acquisition modules. In the 
event of database failure, the workstations can still access their local data, and the 
acquisition modules can continue to receive images. This feature has been used, and 
has also contributed to the high database integrity we have seen. 

The network design of our system is such that it could be converted into a 
fiber optic ring. This would allow other, possibly faster, protocols to be run on it. 
We have also allowed for spare optical fibers in the cable to be used either for 
expansion or replacement of defective links. 

Whatever the topology of the PACS, we think that proper 
planning for equipment failures is a must. A careful examination 
of the system architecture should be done with the thought of 
avoiding a single-point failure bringing the whole system down. 
We have learned that reliability is paramount once the system is 
being relied on in clinical settings. Proper consideration of 
network design may also be used to help prevent obsolescence. 

Fairly simple functions of the database can have far-reaching impact on 
overall system operation. An example is the change made to our mode of operation 
when automatic examination routing was implemented. Even with fiber optic 
connections, the time to transmit a CT examination of 50 slices can be long when a 
user is sitting at the workstation waiting for that study. For this reason, preloading 
the workstations with needed cases seemed to be the best way to reduce this wait. 
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Display from local memory is much faster than over any of the communication 
lines. We had our systems operator make "rounds" on the workstations once or 
twice daily. The list of examinations done that day was used to retrieve the studies 
for each workstation. The neuroradiology CT and MR exams, for example, were 
loaded into their workstation. 

This method was labor intensive, but it worked. Auto-routing eliminated the 
necessity for this procedure. This feature allows the database to send examinations 
to workstations depending on certain selection criteria. For example, all 
examinations of type "US" (ultrasound) are automatically sent to the Abdominal 
Imaging workstation as they are acquired 

The idea of incorporating low-level rule-based "artificial intelligence" is key 
to user acceptance of PACS. A combination of the auto-routing feature with 
improved workstation speed prompted the Ultrasound Section to begin doing 
primary interpretation at the workstation. Prior to this, retrieving all the cases look 
longer than loading the films onto an alternator. A new application for intensive 
care unit workstations automatically displays the most recent previous examination 
when the current image is called up. The expansion of this idea for use in radiology 
should further improve user acceptance. We have done a study of the data 
assembling procedures done in support of radiology reading?, and concluded that 
simple, rule- based selection could equal or exceed the performance of file clerks 
loading an alternator. We will have to determine what the rules are; for example, 
when a CT examination is done, does the radiologist always want to sec the 
previous CT of the same anatomy ? Docs the same hold true if the prior 
examination is an ultrasound or nuclear medicine procedure ? 

These automation concepts should be extended to retrieval from long-term 
storage. When a patient is admitted to the hospital, it is highly likely that he or she 
will have some kind of radiologic study. It is also likely that the radiologist will 
want to see previous films, particularly if they arc of the same area of anatomy. 
The admission should be used to trigger appropriate retrievals from the archive, 
much as is done with film jackets. 

Incorporating concepts of artificial intelligence and 
automation are very important to the growth of PACS* We have 
seen that adding these features makes a very large difference in the 
way the system is both used and perceived. If the PACS simply 
replaces looking at an examination on an alternator, it will be used 
to some degree. If it can improve performance by providing not 
only the requested examination, but correlation studies as well, it 
will find much more ready acceptance. 

Much of the information needed to perform these automated functions 
originates outside of the PACS, Admission, discharge, and transfer information 
usually comes from the HIS. Procedure scheduling and results reporting arc often 
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handled in an RIS. An interface to these systems is an integral part of establishing 
an operational PACS. 

Our HIS includes RIS features, so we are fortunate in having to establish 
only one interface. This effort required about one full person-year, but the result has 
been very useful. At present, the interface is one-way, and the IMACS accepts 
examination requests directly from the RIS. A PC is used both as a protocol 
converter and filter, since we do not acquire all radiographic studies on the system, 
only pertinent requests are processed and sent to the IMACS. The incoming request 
information is converted into an ACR-NEMA data set for transmission to the 
IMACS, but uses RS-232 hardware and protocol rather than the full ACR-NEMA 
interface. The data set contains a fairly small amount of information, since it does 
not have image data to transmit The RS-232 link is not a bottleneck for transfers. 

The end result of capturing the request information is that it eliminates most 
data entry by the technologists, who now only have to type in the patient name or 
medical record number. The rest of the demographic information is retrieved from 
the RIS. In addition, other data in the request can be used to support auto-routing. 

The next phase of the RIS-IMACS interface operation will be to retrieve 
reports. In this way, the images and reports will be accessible from the 
workstations. We are also interested in having the IMACS accept edit transactions 
from the RIS. The primary interest in these is to correct database entries in the 
IMACS. The most frequent change is that issued to correct the patient name. In 
the ICN in particular, babies may be admitted with no given name, so the system 
shows them as "Doe, babyboy or Smith, babygirl". If they are given names while 
still in the ICN, they still retain the same medical record number, but the HIS 
updates the name file. This causes problems when received by the IMACS, which 
will flag an error unless its patient data file has also been edited. This is currently 
done manually. 

The efforts of ACR-NEMA Working Group VIII, which is 
addressing standardization of HIS/RIS/PACS interfaces, should be 
most useful in helping solve the problems of such interfaces. We 
would recommend following their efforts, and when it becomes 
available, adopting the standard practice. 



COSTS 

The Georgetown IMACS development was done under a contract with the 
U.S. Army and MITRE Corporation, and was known as the digital imaging network 
(DIN) project During the contract period, which ended in January, 1990, most of 
the costs of the system were covered by the contract. 

As the DIN contract has been concluded, we are faced with the problem of 
how to expand the system to provide further clinical use. We have seen that support 
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from the physicians outside of Radiology is going to be very important to the 
further growth of the IMACS. Our work with the pediatricians has shown us that 
the support can be substantial. While we cannot yet have any of the system 
overhead reimbursed from third-party payers, we can reasonably ask the Hospital, or 
specific departments, to help support hardware located outside of Radiology. We 
still do not know if this idea will be successful, but if the system can provide real 
utility, and if the physicians are vocal enough in asking Hospital Administration to 
support the system, we stand some chance of relieving a part of our operational 
expense. 

What we have seen to be essential is the enthusiastic support 
of the non-radiology users of the system* A PACS which operates 
so as to earn that support will have a better chance of success than 
one which supports only radiologic operations. 



CONCLUSION 

While we have made progress in transforming our PACS into a clinically 
useful system, there have been a number of difficulties to overcome. We think that 
developments in the areas of human factors for the workstations, improved 
automation and artificial intelligence applications, and the growing use of standards 
will all contribute to making the next generation of PACS much more inviting and 
easy to use. 
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Introduction 

The purpose of PACS (Picture Archive and Communication Systems) is to improve 
the storage and distribution of medical imagery. Large format optical archives are 
being developed to contain one terabyte or more of medical image storage. The goal 
is to provide storage of at least two years worth of medical imagery and to serve 
multiple users via Wide Area Network (WAN) and Local Area Network (LAN) 
image transmission and electronic diagnostic workstations. Optical archive search 
and retrieve algorithms (and the mechanical portions of the search) arc relatively 
slow. Also, the volume of each data set is large enough to impair transmission 
throughput on even the fastest LAN or WAN network. The effectiveness of these 
optical archives depends on data storage format, system configuration and on the 
schemes (including compression) which are developed for storage, retrieval, and 
transmission of data. To date, however, little has been published on operational 
protocols which will make an optical archive workable in real-time in a networked 
clinical environment. The goal of our institution is to develop a networked 
"filmlcss" medical imagery environment over the next three years. To this end, we 
have developed a set of operational protocols for archiving and retrieving of medical 
imagery from an optical archive linked, or more accurately bridged, to local area 
networks (LANs) of workstations and integrated with the Radiology and Hospital 
Information Systems. These protocols minimize search time for images (both by 
the system and by human operators), retrieve projected image needs from the optical 
archive during off hours, and relay them to intermediate term memory in the 
appropriate LAN. The protocols decrease access time for the clinician and 
radiologist and decrease traffic on the optical archive for emergency requests. 

Purpose of a Medical Imagery Archive 

The purpose of a medical imagery archive, regardless of mechanism, is to accept 
new imagery, store the imagery securely, and distribute the imagery to facilitate 
patient care. Electronic archives offer several improvements over the current film 
and shelf system including decreased space requirements, decreased loss of imagery, 
more rapid access to imagery, and (possibly) decreased need for archive personnel. 
Although cost is also an important consideration, the relative cost is difficult to 
project because of changing costs for each segment of the archive. Continual 
increase in recurring cost for space, film, and "low-level" archive personnel with 
concomitant decrease in the cost of computers and storage media suggest, however, 
that electronic archives will soon be more cost-effective than the current system. 
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Flow of Medical Imagery 



Medical imagery used for medical care is distributed to patient care, imagery 
acquisition, and conference locations, both within the medical center and at remote 
medical facilities involved in a patient's care. The features of this distribution pattern 
arc shown graphically in Figure h 
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Figure 1 - Medical Center Imagery Distribution - Imagery is requested (gray arrows) 
from various locations within and outside of the medical center, and the imagery is 
distributed accordingly (black arrows). Most of this imagery transfer is scheduled in 
advance through the Hospital Information System network. A smaller amount of 
imagery (shaded sections of the boxes) is requested on an ad hoc basis. The quantity 
of ad hoc requests varies with the location function. Some locations (double black 
arrows) generate additional imagery for the archive. 



The term imagery is used because electronic imagery storage and transmission 
allows us to include imagery in our archive other than that traditionally thought of 
as medical images (x-rays, magnetic resonance scans, etc.) such as real-time 
ultrasound, clinical pathology slides, anatomic pathology slides, retinal 
photographs, etc. 
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The picture is less confusing (Figure 2) if we consider that there are really only two 
types of requests for imagery made from the archive: scheduled and ad hoc 
(unscheduled). 
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Figure 2 - Requests to distribute imagery from a central archive to locations are 
either scheduled thought the Hospital Information System, or as ad hoc requests 
throughout the day. 

Although Figure 2 shows all scheduled requests routed through the Hospital 
Information System, some scheduled requests (conferences, etc.) may go directly to 
the Data Management System. The important point is that scheduled imagery 
retrieval and distribution can be done outside of peak operational hours. 

Volume of Imagery and Percentage of Scheduled vs. Unscheduled 
Requests for Imagery 

As we have noted before t 2 ][3], electronic handling of imagery has some 
limitations. We cannot transport the same volume of data as with film storage. 
For example, a rough calculation shows that our file clerk moving the carts of 
radiographs to the clinics is moving the data at 2,133 megabytes per second. 
Available electronic systems cannot compete with this. Not all of this transported 
data is actually used. The patient's entire film jacket is sent to each location, but 
only a few of the images in the jacket are actually used. 

To estimate the volume of data we will need to send electronically, we analyzed the 
film distribution to each location in Figure 1, and the percentage of requests 
scheduled the previous day. We also determined how many images of those sent 
were actually used at each location. 

Table 1 shows the quantity of imagery in megabytes requested from our archive at 
SFVAMC and sent each day to the locations in Figure 1. We use a combination of 
film and electronic storage and distribution; these figures were obtained using a 
conversion of 6 megabytes per film, which we have shown previously 1 gives an 
accurate projection of electronic volume. 



TABLE 1 



Location 


Sent 


Used 


% Sched 


MB Sched 


MB Unsched 


Conferences 


3,600 


1,080 


95% 


1,026 


C A 

54 


Clinics 


48,000 


9,600 


98% 


9,408 


192 


Imaging Procedures 


33,600 


8,400 


82% 


6,888 


1,512 


Operating Rooms 


2,400 


480 


90% 


432 


48 


Waids 


8,880 


1,332 


75% 


999 


333 


ICUs 


3,600 


270 


50% 


135 


135 


Emergency Rooms 


9,600 


960 


0% 


0 


960 


OLher Facilities 


3,600 


720 


40% 


288 


432 


DIST. SUBTOTAL 


113,280 


22,842 


84% 


19,176 


3,666 


New MC Imagery 


2.400 


2,400 


f\rrf 

0% 


0 


n a f\(\ 

2,400 


New Outside films 


960 


96 


0% 


0 


96 


NEW SUBTOTAL 


3,360 


2,496 


0% 


0 


2,496 


TOTALS 


116,640 


25,338 


77% 


19,464 


5,874 



The second input to the system is new examinations. New examinations for the 
system (Figure 3) represent both exams done at the medical center and outside the 
medical center. If we add the outpatient exams, with comparison imagery, the total 
sent rises to 1 16 Gigabytes, but only 25 Gigabytes (about 21%) was used. 
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Figure 3 - New Examination Entry into PACS 



Several important observations emerge from the study results in Table 1. Note that 
in the distribution subtotal we sent out 113 Gigabytes of data to the various 
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locations, but only 23 Gigabytes were actually used. Overall 84% of the data to be 
called from the archive and transmitted is known the day before (new exams 
obviously do not have to be called up from the archive), although some locations 
like the emergency room requested all of their imagery on an ad hoc basis. If we can 
determine the imagery which will be used, we can limit the short term memory 
requirement in the system for archive and new imagery to about 26 Gigabytes for 
the exams to be distributed 



The Strategies 

With knowledge about how imagery is currently introduced and distributed within 
the system, strategics can be applied to minimize the limitations of an electronic 
system. 



1. 



2. 



Imagery can be grouped on storage units in the archive. Grouping imagery 
decreases the seek time for the system We have shown^W that the best 
method is to archive by a random number assigned to each patient. We 
advocate the last four digits of the social security number which is already 
assigned and acts as an image identifier as well. Although this system can be 
used with a single unit device like a single platter optical archive, a multiple 
unit device (such as an optical platter jukebox) is required for effective clinical 
operations. 

We can take advantage of distributed short term memory by developing a local 
area network (LAN) for each user area.(Figure 4) 
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Figure 4 - Distributed Memory for PACS using LANs 
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This distributed short-term archive plan also frees up the central Database 
Management System for processing of new imagery being added to the system 
and ad hoc requests. 

3. Since 84% of the imagery is requested in advance, this imagery can be extracted 
from the long-term archive at night, and be immediately available in the short- 
term parts of the network. Since the majority of these cases reflect patient 
scheduling in the HIS, instructions can be sent to the PACS and archive from 
the HIS to move the cases into short-term memory. 

4. Only the cases to be used should be moved into short-term memory. Although 
needs and desires of clinicians and radiologists may differ slightly from center to 
center, decision rules can be fairly easily set up so that only the most recent 
examinations with appropriate numbers of comparison images arc retrieved and 
scnt^l. 

Summary 

Electronic archives have great promise to improve medical care by increasing access 
to medical imagery and eliminating lost images. Certain limitations of electronic 
archives can be eliminated or minimized by development of local area networks 
(LANs) and operational protocols to retrieve only needed imagery and to retrieve 
scheduled imagery outside of peak hours, 
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San Francisco VA Medical Center is conducting a pilot project to complete 
conversion to digital imaging services within the next five years. Our Computed 
Imaging Group at SFVAMC has directed efforts toward PACS development 1 " 15 to 
plan for system expansion and guide the evolution of these systems For it to be 
efficient, a large amount of information from the Radiology and Hospital 
information systems must be available to PACS. Some of this information is 
permanent, such as patient identifiers, however a large amount of the information 
used to route, read, and communicate the results of images is transient We present 
various storage and network solutions to provide this information on PACS, 
allowing PACS to improve radiology operations, rather than merely emulating 
current practice. 

Introduction 

PACS offers potential improvement for the storage and distribution of medical 
imagery, but presents some new limitations as well. The large amount of data in 
medical images (a PA and lateral chest radiograph requires about the same amount of 
data space as the first four volumes of the Britannica) means that a patient's entire 
imagery file cannot be transmitted each time a new image is interpreted or an image 
is reviewed. In addition, a large amount of ancillary information is needed for 
effective use of medical imagery in clinical practice. 

Use of Imagery at Four Representative Centers 

To determine current imaging operation methods, we studied practices at four 

medical centers in the San Francisco Bay Area 5 - We then used this data to 
extrapolate the on-line storage and functions required for effective PACS. The 
centers reviewed were a) San Francisco VA Medical Center [SFVAMC], a medium- 
size Federal facility with academic affiliation; b) the University of California 
Medical Center [UCSF], a large university medical center; c) San Francisco General 
Hospital [SFGH], a medium-size county hospital; and d) Merritt-Peralta Medical 
Center, (a medium-sized private medical center. 

At each center, we identified a) the case compilation process, b) distribution of cases 
to reading areas, c) the need for prior imagery, d) prioritizing of images to be read, 
e) ancillary information used in conjunction with image interpretation, f) the 
transcription process and other sources of interpretation information, g) how images 
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were identified as having been interpreted, h) the fate of interpreted examinations, i) 
how images were sent to other institutions, and j) the construction of teaching files. 

COMPILATION OF CASES 

At all centers, cases were compiled in the file room area by matching the new 
imagery with the patient's film jacket or jackets. Stat cases were placed at the top 
of the stack and given priority. Matching was done by using the identifiers on the 
imagery and the patient's jacket Cases from outside the medical center were searched 
separately to identify whether the patient had a file within the medical center. The 
contents of the film jackets varied at each institution. Some institutions had 
separate file areas for plain films, CT, ultrasound, MRI and nuclear medicine. In 
others, the film jacket contained plain films, CT and MRI, but not nuclear 
medicine, etc. 

DISTRIBUTION OF CASES TO READING AREAS 

At each institution, reading areas were separate and cases to each reading area were 
divided according to type of case. At all four institutions, cases tended in general to 
be distributed according to modality: plain film cases to one reading area, CT to 
another reading area, etc. The larger and "more academic" the institution, the more 
subdivided the cases became. At the University of California, San Francisco, for 
example, plain chest x-rays and bone x-rays each went to separate areas. 

NEED FOR PRIOR COMPARISON IMAGERY OF SAME TYPE 
The relation of confidence to number of prior images used to interpret each 
examination is shown in Table 1. 
Table 1 - Confidence Level and Number of Prior Exams by Type 

NEED FOR PRIOR EXAMS 




0 PRIOR 

1 PRIOR 

2 PRIOR 

3 PRIOR 



■ i i i i i i 

Chest Bone G GU CT MRI Angio 



PRIOR STUDIES 

Both radiologists and clinicians tended to use more prior examinations when 
reviewing chest radiographs compared with other types of examinations. If available, 
at least one single previous examination was always, and using additional previous 
examinations was common. For chest examinations, two to three sets of prior 
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examinations were often used by the image reviewers, and confidence levels 
increased with the number of prior exams available. 

PRIORITIZING CASES FOR IN TERPRETATION 

Most images were interpreted on a first in, first out basis. Interrupting the queue 
and moving cases to first priority was common Requests for these exceptions came 
from a specific area, such as the emergency room, on request of a physician as part 
of the exam request, or on an ad hoc basis by telephone or other contact. 

ANCILLARY INFORMATION F QR IMAQg PSTPRPRETATON 
At all four centers, additional information(besides the comparison imagery of the 
same type was used in the review and reporting process. The most common 
information used is outlined in the examples below. 

Recent examinations of the same body part done on a different modality were 
commonly used. Most frequent were referral to chest CT while interpreting a plain 
chest x-ray (and vice versa), and referral to a nuclear medicine bone scan while 
interpreting a bone x-ray (and vice versa). 

Information from the patient's clinical record was commonly used as a adjunct to 
interpretation, for example, pulmonary function tests or patient temperature in 
conjunction with chest x-ray interpretation. The radiologist generally called the 
referring clinician to obtain this information, rather than accessing the clinical record 
directly. 

Information from the study request and/or the Hospital Information System database 
was used in a large number of cases to contact the referring physician or ward staff 
to impart or get further information, or to call the patient back for further images 

TRANSCRIPTION 

At all four centers, interpretations were dictated for manual transcription into the 
Hospital Information System database. No centers used voice activated 
transcription. At all centers, other sources of interpretation information were 
maintained. These included written abbreviated reports on the front of image 
jackets, notes in inpatient charts, and for several modalities and specialty 
examinations, a separate log book with a chronological list of interpretations. 

IDENTIFICATIO N OF INTERP RETED IMAGERY 

Examinations at all four institutions were identified as interpreted by the absence of 
a paper request, a written interpretation on the front of the image jacket and/or a 
written report in the specialty log book mentioned above. 

FATE OF fffTERPRgTFP EXAMS 

At all institutions, standard outpatient examinations were returned to the archive, 
unless the interpreting radiologist had set the case aside for further review or 
consultation. Inpatient exams and specialty examinations (CT, angio, etc.) tended 
to remain on reading boards for 1 to 5 days to facilitate the consultation process. 
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EXCHANGE OF IMAGERY WITH OTHER INSTITUTIONS 

At all four institutions, imagery was exchanged in the form of hard copy, usually 

duplicates. 

CONSTRUCTION OF TEACHKN O AND QUALITY ASSURANCE FILES 
At all four institutions, teaching and quality assurance files were constructed in one 
of two ways. Cases were either flagged at the time of interpretation, or a search was 
made through the specialty log book for cases which met the search criteria. No 
institution had a database search capability for interpretations within the Hospital 
Information System. 

Implications for PACS 

1 . There must be a means to segregate of cases by type and transmit these cases 
automatically to the appropriate reading areas. This distribution scheme must be 
easily tailored to the specific institution and to changes in section activities within 
that institution. 

2. Automatic protocols must exist in the data management system or archive for 
retrieval of only those comparison images which are deemed necessary at that 
institution for image interpretation and/or review. 

3. A mechanism must exist to flag priority cases and put them first in the 
interpretation queue. The physician at die workstation must be able to adjust the 
queue for ad hoc priority requests. 

4. Ancillary information from the patient's file must be readily available at the 
workstation. These include a) other modality studies, b) identifiers for the exam and 
patient, and c) other ancillary information. 

a) Other modality studies: The user should be able to call up other modality 
studies rapidly from the patient's imagery file to assist with interpretation. A 
mechanism should exist to include other modality studies automatically, if 
these studies are commonly used and meet the criteria for inclusion (e.g. for 
chest x-ray interpretation, the most recent chest CT if done within 3 months). 
The criteria should be easily adjustable by the system manager to meet the 
needs of the users at each institution and section. 

b) Exam identifiers: 

1 . Patient name (Last, First) 

2. Patient ID # (National ID #, Country) 

3 . Hospital ID # (if different from 2.) 

4. Patient Date of Birth 

5. Date of exam 

6. Time of exam 

7 . Modality (Generic type, Vendor, Location) 

8. Exam type 

9. Images in this exam 

10. Center where examinations were done 
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c) Qthgr pafent/exam information: 

11. Priority 

a) routine b) stat 

12. Interpretation status 

a) Uninterpreted b) Preliminary interpretation c) Final interpretation 

13. Patient contact information 

a) Current location b) CL address & tel c) Home address & tel 

14. Referring or ordering physician information 

a) name b) contact address/location c) contact telephone 

15. Attending physician information 

a) name b) contact address/location c) contact telephone 

16. Ordering service information 

a) name b) contact address/location c) contact telephone 

17. Area of origin of request information 

a) name b) contact address/location c) contact telephone 

5. The workstation/PACS should allow groups of cases to be developed for 
various clinical functions. These "case lists" must be easy to develop and use, 
but access and/or modification of them must be restricted to certain classes of 
users. These groups or lists fall into the following categories: 

a. transient one-time (ad hoc conferences) 

b. transient recurring (patients 1 imagery for specific clinics and OR) 

c. transient evolving (inpatients for daily rounds) 

d. permanent (teaching file, quality assurance) 

6. The workstation should offer capabilities beyond the standard alternator. 
Possibilities include: 

a. Free text search of all interpretations in the HIS database 

b. On-line image and text clinical and anatomic pathology data 

c. On-line imaging/medical literature 
d On-line teaching file 

e. On-line list of medical center and appropriate telephone numbers 

e. Voice-activated transcription 

f . Routing of exams by user to another user or workstation at the same or a 
different institution 



Implementation Considerations 

Considerations for implementation include interfacing the sources of the raw data for 
the database, adequate RAM and/or rapid access to intermediate and long-term 
archives to hold the data, methods to update transient information associated with 
the imagery, and maintenance of data and database security and integrity. 

SOURCES OF RAW DATA 
Raw data sources include: 

a. Digital imagery 

b. Digitization of non-digital imagery 

c. Radiology information system 

d. Hospital information system 
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e. Other computer databases 

f . Input from users 

RAM AND/OR ACCESS TO INTERMEDIATE/L ONG-TERM ARCHIVES 
Adequate aggregate network RAM must be sufficient to allow rapid access to the 
appropriate information at each workstation. Alternatively, increasing the speed of 
access to long-term archives and data retrieval from associated computer networks 
such as the HIS would give the illusion of on-line data. 

UPDATING OF TRANSIENT PATIENT/IMAGERY ASSOCIATED DATA 
The information which the user receives at the workstation must be correct. 
Transient information, such as patient location, must be updated in a timely 
fashion. 

DATA AND DATABASE SECURITY AND INTEGRITY 
There must be a method of limiting access to certain parts of the database. In 
addition, the ability to modify the database, (e.g. to construct lists, add to transient 
or permanent files) must be restricted to certain classes of users. We may also wish 
to restrict to certain users the capability to generate ad hoc reports and/or hardcopy. 

Problems to Solve 

Problems to solve include: 

1 . Development of dependable and rapid interfaces to import unmodified imagery 
into the PACS. 

2. Duplication of modality-specific workstation functions at PACS workstations. 

3. Inclusion of non-standard medical imagery (e.g. pathology slides, retinal 
photography, etc.) in PACS imagery database. 

4. Automatic updating of all transient flies associated with the imagery. 

5. Easy assignment and modification of user access and automatic image routing. 

6. Transparent interface of PACS workstations to associated information 
networks, including HIS, other text databases. 

7. Continuous user feed-back to evolve system for maximum functionality. 
Summary 

PACS has the potential to expand the utility and availability of medical imagery. 
Although computers and computer storage capable of using medical imagery in 
ways exceeding the limitations of the lightbox system are available now, these 
capabilities must be offered in a fashion which emphasizes ease of use and a short 
learning curve for the technology to be widely accepted. Attention to completeness 
of the database needed for medical imagery use is important. With a complete 
database we can apply artificial intelligence techniques to increasingly automate the 
PACS environment. 
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Abstract: 

In this paper, we present a PACS project and the related 
concepts developed by three teams: a medical group and engineers from 
two technological research centers. 

The bases of our work are the medical requirements expressed 
by the radiologists to improve the radiology department by new 
technological tools. 

Our aim is to end up with the functional multimedia network 
environment with special emphasis put on the user oriented interface 
for the different kinds of working or viewing stations. 

The multimedia aspect involves an association of images, speech 
and text data allowing to store, in an original way, images and 
interpretation together as a non seperable entity. 

During the first step of the project concerning preliminary 
studies and developments, a prototype workstation is being realized 
and original concepts for image compression have been carried out. 

All technological choices are made to keep the environment able 
to follow the evolution of products and standards. 
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1. Introduction 

The need for a Picture Archiving and Communication System at 
the Centre Hospitalier de Luxembourg (CHL) has been expressed 
initially by the physicians of the Radiology Department in 1987. 
Because no commercialy available ready-made system was in agreement 
with the medical requirements of the radiologists and was open enough, 
a collaboration with research centers was decided to develop a home- 
made PACS based on existing industrial standards. 

After describing the environnement of the project and the team 
working on it, our approach of the PACS and the difficulties that can 
rise will be presented. Then the concepts developed for the differcat 
components of the PACS will be discussed. 

2. The hospital and the PACS team 

With a capacity of 500 beds, the Centre Hospitalier de 
Luxembourg is the largest in the country. Since its opening, the CHL 
has been playing the role of reference center for other hospitals in 
Luxembourg. According to this policy, a project of installation of a 
Picture Archiving and Communication System was decided upon the 
request of the radiologists of the CHL. Their arguments for the 
installation of a PACS are: 

- more diagnostic information are extractible from the digital 

image; 

- the retrieval of previous exams is easier; 

- the costs are better controlled; 

- the efficiency of the department is improved. 
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Every year 48,000 patients are treated and 125,000 procedures 
are realized in the Radiology Department. The already available digital 
imaging modalities are one CT Scanner, one DSA unit, several 
ultrasound machines and one nuclear medicine unit (outside the 
departement). A new building is being built to install a second CT 
Scanner and a DSA unit in 1990, as well as a MRI unit. Within the next 
4 years, digital radiology units will replace the existing conventionnal 
devices. 

As more and more modalities are digital and because the 
departement is being split over two different geographic sites, the 
PACS concept becomes very attractive and an image network will be 
necessary. The extension to other departments within the hospital will 
also be considered as clinicians are interested in the use of the PACS. 
The CHL pilot project experience and the PACS knowledge gained 
during the study will help to diffuse such systems to smaller hospitals 
across the country, allowing ISDN teleradiology applications like 
expert consulting (7, 8). 

To satisfy the clinical expectations expressed by the medical 
group, custom-made developments have to be realized. The CHL team 
providing the medical input is composed of members of the medical and 
administrative staves. The scientific partners are the Biomedical 
Department of the University of Technology of Compifegne (UTC, France) 
and the Public Research Center Henri Tudor (CRP-HT, Luxembourg) that 
has R&D activities in the fields of image processing, networking, 
software engineering, microelectronics... . A management committee 
composed of members coming from the CHL, the UTC and the CRP-HT has 
been created. An important task of the comittee is to promote the 
cooperation between the technical and medical communities. The three 
partners also participate in the management group of the european 
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PACS project ISCAMI (Integrated Systems of Computer Assisted 
Management and Manipulation of Medical Images). 

3* PACS approach and working plan 

The PACS project at the CHL is based on the medical 
requirements to improve the Radiology Department by applying new 
technological tools. During the top-down approach, the clinical 
expectations of the PACS and the conditions for optimal implementation 
at medical, technical, financial and organizational levels have to be 
defined. A statistical analysis of the department is being performed to 
obtain a precise view of its activity. Special emphasis has to be put on 
the designing of user oriented interfaces for the radiologists (5, 9). The 
analysis of previous PACS experiences in other countries is also 
important, as well as documentation studies and participations to 
international meetings in this field of research. 

Localized PACS modules are also developed in a bottom-up 
approach. Among the different realizations which will be integrated in 
the final system, the development of prototype work- and viewing 
stations with friendly user interfaces is a major point of our study. A 
ROC study on resolution has been developed and is now at the clinical 
evaluation level. Other tasks concern the acquisition of images from 
digital imaging modalities or from a film digitizer, the study of the 
ACR-NEMA protocol, the installation of a Radiology Information System 
related to PACS, the evaluation of X-ray sensitive CCD camera for 
digital radiology... . 
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Two phases have been clearly distinguished for the realization 
of the PACS project: 

- the phase I composed of several tasks concerning the 
preliminary studies and developments for 16 months, 

- the phase II for the realization of the PACS at the CHL between 
1991 and 1995. 

At the end of phase I, the feasability of the PACS will be 
determined from the results of the different tasks and the comittee will 
decide if the phase II can start. 

At the time the abstract was sent, the phase I was programmed 
to be shorter. Because the collaboration between the medical and the 
technological communities has not been always easy, the project had to 
be redifined and the first phase extended. We hope that the stability of 
the partners will be reinforced by the creation of the management 
comittee. 

4. PACS Components 

4*1. Working and Viewing Stations 

The conceptual study in this field has led to the distinction 
between two types of stations: the workstation for the radiologist who 
produces the diagnosis and the viewing station for the clinician and the 
radiologist for consuting purposes of their patient examinations (9). 
Their features differ greatly when the required processing abilities 
are compared. The radiologist needs image processing tools such as 
distance and density measuring, contrast and edge enhancement, 
reverse gray scale, zoom and some histogram manipulations (3). In 
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order to avoid the development of non-functionnal stations the number 
of these tools must be limited. Powerfull hardware is necessary and a 
Sun 4-330 GPX workstation with a VME Bus will be chosen. Two to four 
high resolution screens (2000x2000 pixels) will be used for 
multimodality purposes and to allow the display of the entire set of 
images necessary to diagnose. 

PC's or Macsintosh II with an additional high resolution screen 
can be used for the viewing stations. 
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At the present time, a user interface using the X-Window 
protocol is being developed on a multimedia Mac II cx with the Unix 
operating system. Figure 1 presents a prototype interface for the 
radiologist. As the image is displayed, the diagnostis can be vocaly 
added with usual keys activated on the screen by a mouse (record, 
rewind, fastforward, play...). The vocal comment (digital form) can also 
be synchronized with a cursor underlying the pathology on the image 
(this possibility also presents some interest for tutorial purposes). 
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After validation of the diagnosis by the radiologist, the image and the 
interpretation will be send on the network to the medical secretary. As 
no efficient speech recognition system has been designed yet, the 
interpretation is synthetized and then typed by the secretary as text 
data. The complete exam composed of images, text and speech is 
digitally archived as a unique entity according to the ACR-NEMA 
standard (1). 

Because the acquisition of the Sun station will not occur until 
phase II, the application will be transfered to a Sony NWS 1750 or a HP 
9000 available at the CRP-HT and connected to the Mac II Cx. The 
transportability is enhanced by the use of standards (Unix, X-window, 
ACR-NEMA, NFS ...). 

The development of work- and viewing stations with friendly 
user interfaces defined with the radiologists is our major concern to 
facilitate the transition from the lightbox to the stations. 

4.2. Architecture and Network 

Figure 2 shows the final configuration of the PACS at the CHL as 
we conceived it. 

A distributed architecture has been prefered with local 
archiving at the imaging modalities. This solution permits a more 
flexible management and increases the reliability of the system against 
breakdowns. The storage units are composed of juke-box and optical 
disks for on-line and long term archiving (2). 

The backbone of the system will be a FDDI network (10) where 
the imaging modalities and the workstations are directly connected to 
provide high speed communications. Local Ethernet or Token Ring 
networks for the administrative management (HIS, RIS...) in the 
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different departements of the CHL will also be linked to the FDDI 
backbone. The HPMS running on the IBM AS/400 mainframe of the CHL 
manages those administrative applications. 




A central database server Sun S-790 will be installed for the 
management of the distributed database Oracle. The patient file 
hierarchy is vertically organized (patient identifiers — > list of 
examinations --> set of images, text and vocal comment per 
examination) and the selection is made with pull-down menus. 
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4.3. Image Processing and Compression 

The necessary image processings for the diagnostic workstations 
have been presented in paragraph 4.1. Because the user is a physycian 
and not a specialist in image processing, the set of tools must contain 
only the indispensiblc ones which have to be defined with the 
radiologist. For the same reason, the processing interfaces has to be 
very simple and friendly with a reduced number of parameters to set 
up. Dedicated hardware (4) will be developed to allow real-time 
processing 

Because large amounts of data are handled in PACS, compression 
techniques have to be implemented (4, 5), This task has to be 
transparent to the user. Dedicated compression is realized depending 
on the modality used and the tcxtural caracteristics of the image. 
Within an image, segmentation algorithms arc used lo discriminate 
different regions needing different compression algorithms and rates: 
the background with little information and the anatomic region which 
can be subdivided into the primary region (where the pathology is 
localized for example) and the secondary region. A ROC study will be 
implemented to complete this work. The different algorithms tested are 
the Discrete Cosine Transform, the Adaptativc Discrete Cosine 
Transform, The Adaptative Block Truncation Coding, the Huffman 
Coding, the Run Length Coding... . The researchs developed by some 
members of our group in this area are presented in a more detailed way 
in a paper submitted at this meeting (6). 
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4.4. Radiology Information System 

A feasability study has been started to develop a RIS based on 
the 4feme Dimension database software which allows to create very 
friendly user interfaces and is open to other databases like Oracle... . 
This work should point out the specifications of the RIS in order to 
integrate this system to the PACS. Of particular interest is the 
planning of patients appointment to prepare data on the workstations 
and permit a rapid access to previous examination results. The 
connection between the IBM AS/400 and a Mac II has to be tested to 
permit reciprocal updating of the HPMS files and the 4feme Dimension 
files. 

4.5- PACS Training Program 

In order to facilitate the acceptance of the PACS among the 
future users at the CHL and to create a PACS knowledge in Luxembourg, 
a continuous training program will be realized. By involving the CHL 
community as much as possible in the developments, this goal should 
be reached and an optimal use of the system realized. Technical 
training for the maintenance of the system by the technical team will 
also be provided. A well-structured documentation work on the 
developments performed has to be continuously written and approved 
by the management committee for this program to be efficient. 

4*6. Standards 

As we already pointed out, standards have to be adopted at 
every level of our work to end up with an open system and remain 
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vendor-independent if possible. The choice of FDDI, Ethernet for 
networking, NFS, TCP/IP for tranfer protocol, X-Window for interfacing 
and running distributed applications, Unix for the operating system, 
ACR-NEMA for image format, ISO/CCITT recommandations for image 
compression, VME Bus for implementation of dedicated hardware for 
the stations... has been made according to this requirement and allow 
the transportability of the different applications developed. 

5. Conclusion 

An overview of the state of the art of the PACS project at the 
Centre Hospitalier de Luxembourg has been presented in this paper. 
Developments are going on and some of our concepts have already been 
brought to concrete realizations in the field of image compression and 
workstation interface. The compliance to the medical requirements, the 
use of standards and the realization of friendly user oriented 
interfaces are our main concerns. 

Although technological problems will probably occur in the 
future, we know from our short experience that human factors can 
interact and have to be considered. A step by step progression with 
well-defined phases is necessary to bring successfully the PACS 
technology into the medical world. 
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PACS Technical Networking Design 



Clinical and Research Radiology Network 
at the University of Florida 



J.C. Honeyman, T Gillespy, III, M. Frost, et al 
Department of Radiology, University of Florida, 
Gainesville, FL 



Individual components of Picture and Archival Communications 
Systems (PACS) are under active development by a number of 
manufacturers. Unfortunately, there are no comprehensive systems 
available with the flexibility required to build a customized configuration for a 
specific Department or environment. The Department of Radiology at the 
University of Florida has taken a step-wise approach to connecting the 
research imaging and computer systems, clinical imaging systems, 
administrative and personal computers, and an imaging archival system in 
our Department. The immediate goal for each component of the system is to 
solve a specific problem or meet a need for the Department. However, the 
eventual goal is the integration of all radiologic imaging modalities. 

Working Environment and Needs Assessment 

The two teaching hospitals associated with the University of Florida, 
Shands Hospital and the Gainesville Veteran's Administration Medical 
Center (VAMC), are located within 400 meters of each other and are 
connected by an underground tunnel. A Magnatom 63 1.5 T MRI unit 
(Siemens Medical Systems, Istin, NJ), owned by the VAMC, is located in a 
separate facility adjacent to the VAMC. This facility also contains a Radiology 
research laboratory containing a 2.0 T imaging and spectroscopy research 
system (CISCO, Freemont, CA). A Signa 1.5 T MRI (General Electric, 
Milwaukee, Wi) , owned by Shands, is located in a different remote facility 
approximately 900 meters from Shands. The Nuclear Medicine division of 
the Department operates two single photon emission computed 
tomography (SPECT) imagers: a TRIAD (Trionix Research Laboratory, 
Twinsburg, OH) (jointly owned by the two hospitals) located in the VAMC, 
and a BIAD (Trionix) in the Radiology Department at Shands. In addition, an 
image processing research laboratory (IPL) is located in Shands that 
contains several general purpose workstations. The two radiology research 
laboratories (at the VAMC and at Shands) are equiped with a total of five Sun 
workstations. The Radiology Department also owns a separate dedicated 
image processing and display workstation, the DMI 3200 (Dimensional 
Medicine, Inc, Minnetonka, MN). The DMI 3200 is used for multiplanar and 
three dimensional image processing of images from the Signa MRI and two 
GE 9800 CT scanners (General Electric).The original needs assessment 
identified seven areas with either immediate problems or requirements for 
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increased productivity that suggested a network and/or PACS solution (see 
Table 1 ). The remainder of this section describes these areas. 



1 Access to Signa images in Shands Radiology 

2 Access to BIAD and TRIAD images and patient data 

in Shands Radiology 

3 Archival of MR and CT images 

4 Access to images and spectroscopy data by both 

research labs 

5 Microcomputer communications 

6 ICU access to portable radiographs 

7 Access to DM I images by both research labs 



Table 1 . Seven functional requirements for Radiology Network 



Radiologists at Shands need access to images generated outside of 
the main Department, especially images from the Signa MRI and the two 
SPECT imagers. Due to the general clinical and research interest in CT and 
MR exams, studies are frequently removed from the Department film library 
and are not returned. Therefore, archival storage of these studies was 
identified as an urgent problem. In addition, the two research laboratories 
require access to images and spectroscopic data from remote sites. 

In addition, Nuclear Medicine needs clinical access to the TRIAD and 
BIAD images with viewing and analysis capabilities on a radiologist's 
workstation. Reconstruction of SPECT images, initial analysis, and filming 
was usually performed on a remote disk-less workstation, with further 
analysis and research performed on the radiologist's workstation 

All radiologists, scientists, and secretaries in the Department have 
either an MS-DOS or a Macintosh computer in their offices for producing 
research papers, proposals, and correspondence. Cooperative research 
requires rapid interpersonal communication and the ability to easily transmit 
documents and data to others in the Department. With the faculty and staff 
widely dispersed, communication among the groups was identified as a 
problem area. In addition, radiologists also need to be able to input 
radiologic images on their microcomputers for incorporation into teaching 
and presentation materials. 

Approximately 50,000 portable radiographs are produced per year in 
the five intensive care units (ICUs) at Shands Hospital. Unfortunately, 
clinicians do not have timely access to radiographs obtained in acute care 
situations since none of the ICUs are close the Department. Not 
surprisingly, these radiographs were frequently removed from the 
Department without proper signout and were subsequently lost without an 
official interpretation. 
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Imaging Network Design 

RADNET is comprised of a fiber-optic cable ring with clusters of 
individual nodes connected via thinnet coaxial cable (figure 1). The thinnet 
coaxial cable runs ethernet and TCP/IP protocols, while a proprietary ring 
protocol runs on the fiber-optic cable. The connection between the thinnet 
and fiber-optic cable is made through WhisperLAN gateways (FiberCom, Inc, 
Roanoke, VA), with a repeater at each gateway and an extra repeater at the 
center of the installation. Currently, no subnets have been established. 
RADNET allows the two research laboratories to access images from the 
Signa and Magnatom MRIs, the DMI 3200 workstation and the SPECT 
imagers. Images from the 9800 CT scanners and other devices can be 
loaded into the DMI 3200 from magnetic tape and then made available to the 
network. The Nuclear Medicine component of the network allows patient 
lists and images from both imagers to be displayed on the TRIAD, the BIAD, 
an analysis workstation, and a radiologist's workstation. 

RADNET operates at a theoretical rate of 10 Mbps. Typical transfer 
rates during average loading of the network are in the range of 1 .5 - 2 Mbps. 
Performance of RADNET is adequate when the only active nodes are in 
Nuclear Medicine. When image' data is transferred to or from either research 
laboratory while Nuclear Medicine is active, performance is significantly 
degraded. 

Microcomputer Network Design 

The Department personal computers are interconnected via an 
AppleTalk network. The AppleTalk network is comprised of a PhoneNET 
StarController (Farallon Computing, Berkeley, CA) connected to each 
computer through unshielded twisted pair cabling. TOPS network software 
(Sun Microsystems, Alameda, CA) on the personal computers and one of 
the Sun computers in the IPL allows printer sharing, print spooling, file 
transfer among all three computer types, and the ability to mount and publish 
volumes for sharing hard disk resources. Electronic mail is performed by 
InBox (Sun Microsystems), which has versions for both the MS-DOS and 
Macintosh computers. A FastPath 4 gateway (Kinetics/Excelan, Walnut 
Creek, CA) connects the AppleTalk and ethernet portions of the network. 

The AppleTalk network operates at 230.4 kbps. Performance has 
been adequate to date with 10 Macintosh, 22 MS-DOS and 1 LaserWriter 
Plus active nodes. 

ICU Teieradiology 

Portable images are transferred to the ICUs by a dedicated 
teieradiology system, the Clinical Review System (CRS) (Dupont Medical 
Division, Wilmington, DE). The CRS is composed of a laser digitizer and 
server module in the Department which digitizes the portable radiographs. 
The server transmits the digitized images to one of 5 display stations in the 
ICUs over thicknet coaxial cable using a modified ethernet protocol. An 
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additional display station is located in the image processing laboratory. Each 
display station has a 160 Mbyte hard disc capable of storing 120 radiographs 
digitized at 1024 x 842 x 12 bits resolution. The CRS is not presently 
connected to RADNET. 
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Figure 1 : Initial Network Configuration 



Future expansion 

The original network and teleradiology systems meets all our 
requirements except the archival of MR and CT images. Expansion of 
RADNET is planned to allow direct digital acquisition of conventional 
radiographs, archival of images from all modalities, and advanced display and 
analysis capabilities. In addition, the teleradiology functions will be 
incorporated into RADNET. 

Our design activities include identification of locations of work areas for 
radiographic interpretation and consultation, the location of bridges to create 
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subnets for traffic load containment, the solution to problems of file and 
protocol conversions for equipment integration, and identification or 
development of display devices to meet the varied needs of end users. The 
Department is separated into logical, physical and functional clusters (table 
2). The logical clusters generally define divisions or subspecialties in the 
department and are used to identify locations of display and analysis 
workstations for interpretation and consultation. Physical clusters describe 
the equipment in terms of location for the identification of possible subnets 
to reduce overall network traffic. Functional clusters define groups of similar 
image modalities and are used to specify unique display, storage, and 
analysis requirements. 



Clusters 


Examples of Clusters 


Logical 


Neuroradiology 
Abdominal Imaging 
Musculoskeletal 
Radiology 


Physical 


Locations of Equipment 


Functional 


MRI 
CT 

Ultrasound 
Nuclear Medicine 
Radiographs 



Table 2. Examples of Design Clusters in Radiology 



Neuroradiology has been identified as having the highest priority for 
archival of MRI and CT images. We have chosen a Medical Imaging Gateway 
(Vortech Data, Inc., Richardson, Tx) as the center of a communication 
management, archival database, and image display system. Long term 
archival will be through the Vortech IARS Image Archival Retrieval System 
with dual 4.0 Gbyte Optimem optical disk drives, a 131 platter optical 
jukebox, and 2.0 Gbyte magnetic working disk. The system will initially 
archive neuroradiology images from the Signa and two 9800 CT scanners. 
The medical imaging gateway will interface with the DMI 3200 and the 
research laboratories through the existing RADNET, and with two dual 
monitor Imlogix 1000 Display Stations through a Fiber Distributed Data 
Interface (FDDI) protocol. The network has been expanded to include 
bridges internal to RADNET and a router for connection to Medical Center 
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and University networks. The new network configuration is based on the 
original network with additional components (figure 2). 
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Figure 2: Updated Network Configuration 
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Summary 

Initial evaluations of RADNET, teleradiology and the AppleTalk 
network indicate that clinical radiology networks are feasible and can result in 
increased productivity, improved patient care, and potential cost reductions 
through resource sharing. Our design experience has demonstrated the 
importance of a correct initial network configuration with planned expansion 
capability. 



Hospital Integrated Picture Archiving and Communication 
(HIPACS): A European Project 

R. Mattheus, F. Moyson, Y. Temmerman, et al 
Department of Radiology, University of Brussels, 
Brussels, BELGIUM 



In recent years, the field of medical imaging has gradually shifted towards digital 
techniques.The management of the large amount of medical images and information 
generated by today's clinical services is a growing problem. For the management of this 
enormous amount of images a PACS environment can be set up. But technical, 
economical and organisational problems have limited the exploitation in daily clinical 
routine work. The main goal of the HIPACS project is preforming a platform for a second 
generation PACS concept.The second generation PACS concept will be a multi-vendor 
environment integrated in the hospital. A phase upgradeable approach will be chosen so 
that a stepwise introduction can take place; this is also more realistic from an economical 
and organizational point of view. Universities, industry and hospitals of Europe are 
working together to perform this task. The paper will give a brief overview of the project 
with a focus on networking at the university Hospital of Brussels (VUB). 

1. INTRODUCTION. 
1.1. HIPACS. 

In June 1989, HIPACS became an official CEC (Commission of the European 
Communities) sponsored project, with as its Prime Contractor the PRIMIS Research 
Centre. Michel Osteaux became project leader, Rudy Mattheus project manager. 
Ten partners from five European countries joined forces in this project and brought 
together complementary skills in medecine and information technology (IT). Their 
origins lie in the industry (Philips Hamburg; BAZIS Leiden), in hospitals (Klinik 
fur Radiologische Diagnostik Aachen - Univeristy Hospital VUB Brussels) and in 
universities (University of Ulster - Institute for computer Science Crete- Lehrstuhl 
fur Messtechnik RWTH Aachen - PRIMIS Vrije Universiteit Brussels). PRIMIS has 
two subcontractors, B ARCO and GE-CGR. 

The aim of this project is the identification and, whenever possible, the elimination 
of bottlenecks that hamper the implementation of a full-scale Hospital Integrated 
PACS. In order to achieve this goal, coordinated research efforts in the field of 
digital network design, data management, and diagnostic imaging workstations are 
required 
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1.2. HIPACS project topics. 

The modelling and the related coupling ofPACS and HIS 

When introducing an image management level, an important point in using 
the PACS in clinical routine is its integration with the patient management system 
of the hospital, the Hospital Information System (HIS) and Radiology Information 
System (RIS). Patient information must be shared between these three levels. 

A prototype of a high-speed network for image transfer . 

For transporting this enormous amount of information classical networking 
is not adequate. To solve this problem work is being done in the direction of optical 
networking. Fast image communication techniques axe being studied, developed and 
evaluated. 

A model of a hetrogeneous network structure will be used to study routing 
optimization of image and other data in existing and future networks. 

Knowledge based network management techniques are being developed, so 
that it will be possible to route these images to places where they are normally 
needed. These techniques are based on the organisation and medical information of 
the Radiology department. The development of these tools started two years ago in 
our department. 

Multimedia data-base software with facilities for intelligent information retrieval 

Images, patient data, electrocardiograms, all these multimedia data must 
be considered for storage and retrieval. Retrieval mechanisms based on the 
information contained in the images are also an important topic. 

An adaptive user-interface for increased diagnostic efficiency of existing image 
workstations. 

Medical knowledge is not only important for network management; it is 
also useful information for defining the required behaviour of the user-interface. 
Suggestions will be made as to which images should be displayed where on the 
reporting stations. 

2. DISTRIBUTED NETWORKING. 
2.1. Introduction. 

One can choose very fast networks in order to try to solve the image communication 
problem. In the AZ-VUB Ethernet was chosen for communication. 
This requires the setting up of a network topology and management, which can 
render the whole system more acceptable for the radiologist and which allows for 
better work than the currently used manual method. Using this, the intention is to 
build a distributed PACS archiving system, which has a management system based 
on an expert system. The concept is that images, of which the radiologist foresees 
that he will need them the next day (e.g. images of patients having an appointment 
for that day) are fetched from the archives the night before and are stored in the 
radiology department's own local memory, so that the radiologist can consult 90% of 
his images as it were instantaneously. Only the images of which he had not foreseen 
that he would have to use them, will have to be waited for, but if these had to be 
sought manually, he would probably have to wait for a considerably longer time. 
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2.2. Phased build-up. 

The AZ-VUB project is executed in distributed upgradeable phases. In a first phase, 
PACS systems are set up at image captation on a limited scale: a local network with 
acquisition, display and storage, and this for each type of image generating 
equipment. The CT-PACS consisting of 2 CT units (Siemens) a prototype 3 screen 
reporting station (Siemens), local storage and archiving possibilities. A hardcopy 
unit (3M) used for the 2 CPs and reporting console. The MR-PACS section 
consists of 1 MR (Siemens) and a powerful image-processing station and local 
storage and archiving possibilities. At the emergency department the PACS section 
consists of a laser-digitizer, a 2-screen reporting console and a prototype DR (II) 
(CGR-GE).In the middle of 1990 a test at the orthopedics department with an 
Imlogix viewing station will be set up. Also a project with 3M has been set up to 
manage ultrasound images. Together with Barco, a local PACS environment was 
built for the management at the intensive care unit of images generated at the 
bedside. 

These subsystems are linked via Ethernet to a backbone network in the radiology 
department. This backbone network consists of a buffer storage, optical disks, a 
PACS reporting console and a hardcopy unit (3M). The network currently in use 
(MR, CT and radiology) has a length of 300 metres. 

At the moment, an optical fibre network with a length of 1,5 km is being 
completed. This network consisting of some 20 parallel fibres will not only be used 
for PACS, but also for a number of other hospital informatics projects. It will allow 
for further extension of the PACS system towards the nursing units. Experiments 
are also being considered which will extend outside the hospital walls (e.g. towards a 
radiologist on duty, who can then, during the week-ends, give advice, based on 
images sent to his home, to less experienced radiologists inside the hospital). 

2.3. Network control. 

Control over a network can be centralized or distributed. Centralized control means 
that one of the devices on the network has absolute control over who can use the 
network at what time. 

In a network with distributed control, none of the system nodes has explicit control 
over the network. Instead, control is distributed evenly among all members in the 
network. The merits of distributed systems include greater reliability, simpler bus 
structure, and more flexibility on system configuration than the centralized systems. 
However, they introduce larger overheads and therefore reduce the effective channel 
bandwidth. Nevertheless, the distributed systems are still the most popular for 
currently implemented PACS. 

ETHERNET is a typical example of a distributed control network. Each node is 
attached to a single high-speed coaxial bus by a connector clamp, a transceiver and a 
controller. The transmission medium may be occupied by only one system 
component at any one time instant. As a result each node has to compete for use of 
the common bus under a decentralized control strategy known as carrier sense 
multiple access (CSMA). Under this scheme each node listens to the carrier before it 
attempts to transmit data. If there is no carrier already on the bus (idle), it transmits 
its data. Otherwise, it waits for a random period of time before its next attempt. 
ETHERNET uses coaxial cable as a transmission medium. Its theoretical 
transmission speed is rated at about 10 megabits per second. However, because of 
overhead in the system, the effective transmission rate is only about 1 megabits per 
second 
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3. NETWORK TOPOLOGIES FOR PACS. 



It is possible to define 4 main PACS topologies: closed line, central line, distributed 
network and distributed subnetwork. 



3.1. Closed line. 



A closed line is modality-dependent and only aimed at connecting devices of the same 
modality (CT, MR,...) together, possibly, with a second console. A typical 
construction will contain two modalities of the same type connected with an image 
console. 



3.2. Central line. 



Added to the first topology is a central computer system with a local store, data base, 
archiving possibilities and display and research software for the reporting node. It is 
possible to interconnect devices of different modalities. 



3.3. Distributed network. 



Added to the previous topology is the possibility to make use of distributed 
processing on the different computers in the network. 



3.4. Distributed subnetworks. 



Each sub-department (MR Department, CT Department, Emergency Department,...) 
has its own small PACS environment with local store, reporting consoles, hardcopy 
units and a gateway or bridge to the PACS-backbone network which interconnects 
all the sub-networks. This approach allows distributing the image-flow over the 
network, making traffic on a single subnetwork possible without interfering with the 
traffic on the other subnetworks. Figure 1 gives an example of this kind of network. 
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Figure 1 

An example of a distributed PACS subnetwork. 
The three departments on the figure are interconnected 
through the PACS-backbone. 
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4. THE NETWORK MANAGER. 

4.1. Introduction. 

The network manager shell insures that traffic on the network is optimal. Buffering, 
decentralization and pre-fetching are techniques used to allow faster access of 
examinations (possible delays are due to image transport). 

During the night, a link between the Radiology Information System (RIS) and 
PACS allows the search in the archives of all images of patients scheduled for 
control or new examinations the following day. This procedure is called prefetching. 
All network management techniques have been developed in-house. A program data- 
base is used to maintain, change and further develop these programs. This is 
essential because of the fact that a lot of people are involved in the project. 
PACS has a hierarchical architecture introducing several levels for image transport, 
control, security and management. The system is distributed according to the image, 
data, hardcopy and control level. 

Terminals are integrated on the different acquisition modalities containing the control 
functions to request image transfer. On each modality the radiologist can select 
different functions: archive, report, special study and temporary storage. The 
messages on the control layer are stored on queues managed with different priorities. 
The image-manipulation and reporting console communicates on image-transfer, 
control and security level. The communication with the optical disk-image data-base 
takes place via message structure, in time-sharing mode on the control level and by 
batch processes created by the control layer on image-transfer level (making use of 
statistical information). Images are converted to the SPI (ACR-NEMA) format. The 
network management and security-level is centralized. Any existing data-base system 
may be used to link images to non-image information. The Standard Product 
Interconnect (SPI) standard (an ACR-NEMA superset) is used for image format and 
network commands. 

The management level performs the logical management of the stored images and 
takes care of all supervisory and administrative tasks. In addition it performs the 
logical link of image to related non-image information such as patient data. The 
security level forms the top layer, which supervises the different tasks, notes the 
history of events and controls login and password information. 
This hierarchical structure of the network allows the workload of the network to be 
optimized and enables us to handle image transport 24 hours a day. 
The network shell uses different input parameters, correlations between the workload, 
report and image generating functions at the radiology and emergency departments 
(amount of images per hour or day). By processing this information and other 
statistical data (appointments, priorities,...) we have a constant continuous 
throughput function instead of a highly variable discontinuous function, and are able 
to use our PACS environment in clinical practice. 

4.2. Simulation of the network. 

A prototype of a network management system has been implemented (Figure 2). Its 
different components are a network model, a simulator, traffic and flow managers and 
configuration and strategy managers. The physical link between this simulation and 
the actual PACS network has not been realized yet, but it is our aim to use the 
different managers to optimize the actual network in real-life situations. 
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Figure 2 

An overview of the Network Management Simulation System. 



A graphical editor allows the interactive specification of a specific network 
configuration, i.e. a composition of network segments with associated image 
acquisition stations, viewing stations, storages, hardcopy facilities, etc. The 
combination of the generated network model and the simulator allows us to test and 
experiment with different traffic management strategies and different network 
configurations. 

Two main modules of the network management system implement the traffic 
management task. Briefly, the flow manager creates and alters image transfers, while 
the traffic manager sequentialises those image transfers over the network. 
A fundamental requirement is to make the traffic management strategy and the 
domain knowledge used in it, explicit. To fulfil this objective, both the flow 
manager and the traffic manager make extensive use of strategy-rules, which are the 
explicit representation of bits and pieces of common sense and/or domain knowledge 
related to managing the PACS network. 



4.3. Traffic manager. 



The function of the traffic manager is to sequentialise the image transfers over the 
network according to a selected traffic strategy. The implementation of this module 
is distributed, i.e. on each segment of the network a single traffic manager is 
assumed. 

A hierarchy is defined among segments. Each segment is connected to its only 
supersegment by means of a bridge. The highest segment in the hierarchy, i.e. the 
segment which has no supersegment, is called the backbone of the network. In 
accordance with this, the traffic manager of a segment's supersegment is referred to as 
the supervisor of this segment. 

Traffic managers receive transfer requests, i.e. requests from image stations to 
transfer an image, originating from stations located at the traffic managers' segments. 
More details of such requests (e.g. the source, destination and priority) were filled in 
by the flow manager. 

These completed transfer requests, also called transfers, are stored on transfer queues. 
All logic storage units such as image stations (logic storage units will be explained 
later on) and traffic managers have transfer queues. A queue of a traffic manager is 
called the queue to its supervisor. The decision on which queue to store a transfer is 
made according to the queueing strategy of the traffic managers. 
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For each image transfer one traffic manager, the so called master of the transfer, is 
responsible for activating the transfer. The master of a transfer is also set according 
to the traffic manager's queueing strategy. 

A traffic manager 'manages' the queues of modalities on its segment and the queues 
of traffic managers on its subsegments (i.e. the segments lower in the hierarchy 
which are connected to this segment with a bridge), i.e it is responsible for serving 
the transfers of these queues in a correct order. The queueing strategy of traffic 
managers defines the way in which a transfer is put on a queue which is managed by 
the transfer's master. 

Traffic managers constantly investigate whether transfers waiting on the queues 
which they have to manage are allowed to be executed and consequently interrupt 
other running transfers. The traffic managers' interrupt strategies define different 
solutions (mostly based on the urgency of different transfers) to deal with this task in 
a satisfactury way. 

4.4. Flow manager. 

The flow manager is responsible for many tasks. One of the tasks of the flow 
manager is to decide where an examination must be stored at each time and to 
distribute images over the network's storages to minimize network traffic for later 
retrievals of the examination. The flow manager also has to ensure that urgent 
transfers are served sooner than less urgent transfers, and that reasonable response 
times are achieved for all transfer requests. 

Further tasks of the flow manager are organising prefetches at non-peak hours to 
minimise network load at peak hours and monitoring the load factor of the storage 
media on the system to prevent overflow. Just like the traffic manager, the 
implementation of the flow manager is distributed, i.e. on each segment of the 
network a single flow manager is assumed. 

Flow managers receive transfer requests from stations located at the flow managers' 
segment. In turn, they send completed transfer requests, i.e. transfers, to the traffic 
manager of their segment. Different types of transfers are defined. A transfer request 
originating from an acquisition station will result in an acquisition transfer, one 
originating from a consultation station will result in a consultation transfer and one 
from a reporting station in a reporting transfer. Other existing transfer types are copy 
transfers and copy-bef ore-delete transfers, which copy or move examinations to other 
storage devices. 

Flow managers assign a priority level to a transfer. Knowledge about types of 
transfers, their importance, their frequencies, etc., is necessary to make such a 
decision. Some very important transfers receive the status of emergency transfers. 
Priorities and the status of emergency transfers are assigned to transfers to make it 
possible for the traffic manager to differentiate between urgent and less urgent 
transfers. 

To decide where an examination must be stored at each time in order to minimize 
network traffic for later retrievals of the examination, the flow manager needs 
knowledge about patient-flow and examination-flow. Supported by this knowledge, a 
flow manager assigns a source and destination to a transfer. Examinations can be 
sent to or taken from logic storage units. These are logical storage devices which 
group sets of physical storage devices (i.e. disks) which all store examinations 
according to a common purpose. Logical storage units are the image stations 
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(acquisition, consultation and repenting stations), the local stores, local archives, 
central stores and the central archive. Local stores group devices which have as main 
purpose to store the acquisitions made by acquisition stations on each segment. 
Local archives archive examinations which are expected to be needed in the near 
future by modules on each segment. Central stores store examinations which are 
unlikely to be needed in the near future and the central archive archives examinations 
which are of no interest in the near future (e.g. after a period of 6 months an 
examination can be stored in the central archive). 

To distribute images over the network's storages to minimize network traffic for later 
retrievals of the examination, copy transfers are created by the flow managers. To 
ensure that reasonable response times are achieved for all transfer requests, flow 
managers adjust the priority value and emergency status of waiting transfers. Also 
adjustment of source and destination of transfers is a task of the flow managers. 
Monitoring the load factor of the storage media on the system to prevent memory 
overflow can be done in different ways, as will be described in the next chapter. 
Organising prefetches at non-peak hours to minimise network load at peak hours 
needs the hospital appointment system and common sense. At this moment, 
prefetching is done manually in the prototype. 

4.5. PACS strategies. 

Strategy-rules are the explicit representation of bits and pieces of common sense 
and/or domain knowledge related to managing the PACS network. Strategy-rules are 
grouped in rule-classes which correspond to a strategy for handling a specific subtask 
of the management task. Often alternative ways for handling the same management 
task are available. The corresponding rule-classes are in turn grouped for 
classification purposes. 

Two modules make extensive use of strategy-rules, i.e. the flow-manager and the 
traffic-manager. Selecting within each of these modules a strategy for each of the 
subtasks involved results in the specification of a particular flow-strategy and traffic- 
strategy respectively. A traffic-strategy and a flow-strategy together specify the 
overall strategy of the network manager. Strategies interact, they can not be 
combined arbitrarily. 

The strategy manager is there to evaluate the performance of the network and to 
advise on possible improvements of the strategy employed. It does this by analysing 
the logs generated by the simulator. It contains expert knowledge on how certain 
types of problems can be avoided. The proposed changes can then be evaluated by 
editing the traffic and flow manager module, running the simulator and again 
analysing the resulting logs. 

The configuration manager evaluates the performance of the network and advises on 
possible improvements of topology and configuration. Just like the strategy 
manager, this module analyses the logs generated by the simulator. It also contains 
expert knowledge and its suggestions are also checked by running a new simulation 
with the necessary changes. 
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5. CONCLUSION. 

Modular network architectures, capable of reacting to growth and changes in traffic 
patterns are essential. Partitioning the network into patterns is essential as well. 
Partitioning the network into subnets will reduce bandwith requirements and increase 
response time by distributing network traffic. A hierarchical decomposition will 
make it easier to use transmission techniques with an optimum choice on each level. 
As an additional benefit, this design strategy offers the possibility to start 
introduction of PACS into a given environment by establishing the local sub- 
system first. In subsequent steps, the integration of a complete PACS could be 
performed, accompanied by a step-by-step training of personnel and a gradual change 
of work routines. In the future new techniques for faster communication will be 
available, but there will probably always be an advantage in management of image 
distribution in a clinical environment. 
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ABSTRACT 

The ACR-NEMA Standard was developed initially as an interface standard for 
the interconnection of two pieces of imaging equipment. Essentially, the Standard 
defines a point-to-point, hardware connection with the necessary protocol and data 
structure so that two differing devices which meet the specification will be able to 
communicate with each other. 

The Standard does not define a particular PACS architecture, nor does it 
specify a database structure. In part, these are the reasons why implemented have 
had difficulty in using the Standard in a full PACS. 

Recent activity of the Working Groups formed by the Committee overseeing 
work on the ACR-NEMA Standard has changed some of the "flavor" of the 
Standard. It was realized that connection of PACS with hospital and radiology 
information systems (HIS and RIS) is necessary if a PACS is ever to be successful. 
The idea of interconnecting heterogeneous computer systems has pushed Standards 
development beyond the scope of the original work. Teleradiology, which 
inherently involves wide-area networking, may be a direct beneficiary of the new 
directions taken by the Standards Working Groups. 

This paper will present a brief overview and history of the "parent" standard, 
describe the structure of this standard, and discuss the "offspring" of the standard 
with regard to the activity of the Working Groups. 



INTRODUCTION 

Though the concept of PACS has been well embraced by the medical 
imaging community, this has been so largely on a small-scale basis despite the fact 
that satisfactory technology for implementation has been available for several years. 
Though perhaps the major reason for this slow adaptation is the cost of systems, the 
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inability to connect multiple- vendor imaging departments to networks (or network 
interface units, NIUs) is also a contributing factor. 

This problem, the lack of hardware and software standards for image 
interchange, prompted the formation of the ACR-NEMA Digital Imaging and 
Communications Standards Committee. The original goal of the Committee was to 
produce a standard which would allow imaging equipment from different 
manufacturers to exchange images and associated data. 



HISTORY 

At the first U.S. PACS meeting, held in 1982 in Kansas City, acquiring 
image and image descriptive data from different manufacturers' equipment was 
already seen to be a problem. 1 Most groups doing PACS-related work were 
acquiring images for their systems either through digitized video (which suffers from 
limited dynamic range) or magnetic tape. The latter method meant understanding the 
tape format and pixel packing technique, something the manufacturers were not 
always willing to reveal. The potential users of PACS, the radiology community, 
asked the American College of Radiology to help find a solution to this problem. 
Engineers who worked for the manufacturers also began to realize that there was 
little to be gained by keeping image formats proprietary. The manufacturers were 
represented in issues of standardization by a number of groups, but their 
Association, the National Electrical Manufacturers Association (NEMA) had 
experience in standards related to medical imaging equipment. The two 
organizations first held exploratory talks in late 1982, and the result was the 
formation of the ACR-NEMA Digital Imaging and Communications Standards 
Committee in early 1983* 

The original charter of the Committee was to develop an interface for the 
exchange of image and related information between a piece of medical imaging 
equipment and whatever was on the other side of the interface ^. in defining this as 
its charter, the Committee restricted its range of activity. The resulting standard 
would not be a standard for a full PACS, a database, or a network. However, the 
ability to bring digital image information out of a piece of equipment over standard 
hardware using a standard format was felt to be a goal worth persuing. 

The Committee formed three Working Groups, each addressing a particular 
aspect of standardization. The Working Groups were assigned to hardware and 
protocols (WG I), data format (WG II), and system and performance specifications 
(WG III). Coordination of the Working Groups was assured by both Committee 
action and the participation of members in more than one group. 

The Working Groups began their tasks by examining existing standards since 
it was the intent to use other standards if possible. Hardware standards examined 
included IEEE 488, Ethernet, Intelligent Peripheral Interface (IPI), SCSI, DEC® 
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DR11W, DEC Unibus®, and Multibus. Data formats of AAPM Magnetic Tape, 
LANDSAT, NATO magnetic tape, and the Japanese JAMIT were examined for 
possible useful data structures. While some ideas were drawn from these standards, 
none in particular was satisfactory for all the requirements defined. In addition, 
because of antitrust regulations in the United States, NEMA must operate under 
certain constraints. This prohibited the use of certain proprietary interfaces, or 
interfaces which would unfairly benefit a single manufacturer. The effort of 
examining standards did not end when the Working Groups concluded that a new 
specification would have to be written. The MAP/TOP standard, for example, was 
discussed even after the ACR-NEMA draft was written. 

Because of rapidly evolving PACS research, it was thought necessary to get a 
standard written as quickly as possible. The Working Groups went through 
numbers of proposals by circulating them to each other and the engineering staffs at 
the parent companies. Within two years, a standard had evolved which was thought 
to meet all the requirements set forth by the Committee. Following Committee 
approval, the document was approved by vote of the member companies and ACR 
representatives. The approved draft was distributed at the Radiological Society of 
North America meeting in 198S and published as a NEMA Standard (ACR-NEMA 
300-1985) in the same year . 3 



ACR-NEMA 300-1985: THE PARENT STANDARD 

This Standard, titled "Digital Imaging and Communications" includes the data 
format, protocol, and hardware interface necessary for transmission of data between 
two pieces of medical imaging equipment. The location of the interface plane is 
defined to be between the two devices as shown in Figure 1 . 
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FIGURE 1 : LOCATION OF THE ACR-NEMA INTERFACE PLANE 

Though not intended as a network standard, it was certainly anticipated that 
for use in a PACS environment, connection to a network of some form would be 
essential. In this case, the device on one side of the interface would be a network 
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interface unit (NIU), the network in use then determining the nature of the NIU. It 
was recognized that such a unit would have to have significant local processing 
power, probably on the order of a microcomputer. However, given the magnitude of 
the interfacing problem, this was thought to be a reasonable approach. A diagram 
of how the ACR-NEM A interface might be used in a network environment is 
shown in Figure 2. 
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FIGURE 2: LOCATION OF ACR-NEMA INTERFACE PLANE 
WITH A NETWORK 



In designing the interface, the Working Groups used the International 

Standards Organization Open Systems Interconnect (ISO-OSI) Reference Model. 4 
In this model, the functions of a network communications interconnection are split 
into well-defined layers. Each layer performs a specific set of operations, providing 
services to the layer above and below. On each side of an interface, the same layers 
are performing the same functions, so to the programmer, the layers can be thought 
of as communicating directly with their equivalents. This "peer-to-peer" 
communication actually occurs in a vertical direction in the layers, and the true 
connection is only at the physical layer. 
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The ACR-NEMA Standard builds upon the ISO-OSI Reference Model in that 
specific layer functions were defined. The ACR-NEMA document refers to named 
layers and divides functions accordingly. An attempt was made to adhere to the 
layer functionality of the ISO-OSI model, but the ACR-NEMA Standard is not ISO 
compatible. In addition to differences in definitions of layer-to-layer connections 
(called interfaces in ISO terminology), there is one convention used in the ACR- 
NEMA Standard which violates an ISO-OSI concept (this is discussed later). The 
correspondence between the ISO-OSI layers and ACR-NEMA layers is shown in 
Figure 3. The discussion that follows will start at the topmost (or Application) 
layer of the Standard and proceed downward to the physical layer. 
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FIGURE 3: SIMILARITY OF ISO AND ACR-NEMA LAYERS 

When medical imaging studies are performed on a patient, the demographic 
information about that patient, the data on how the images were acquired, and the 
images themselves form a set of data amenable to display and interpretation (a data 
set in ACR-NEMA terminology). In the Standard, this data set is further associated 
with commands, and the whole is called a message. The commands indicate certain 
fundamental operations on the data: SEND, GET, FIND, and MOVE. SEND is the 
basic command, allowing an imaging device to output its data. GET allows a 
requesting device to obtain a specific data set FIND is a primitive search tool 
which will locate all data sets which match the information requested. MOVE 
allows one device to request a second to send data to a third. In the Standard, 
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assembly of the message from individual data items and commands, and formatting 
of the result of this assembly takes place in the Application and Presentation layers. 
The aformentioned problem with OSI compatibility results from the upper layers, 
which build the message from commands and the data set, making a distinction 
between data types. Commands and data travel in different types of packet, but this 
is not encoded until the lower layers. This means that the upper layers have to look 
into the packet to determine the type upon receiving a packet, or have to tell the 
lower layers whether what is being passed down is command or data. One of the 
areas of investigation by the Working Groups is how to either eliminate this 
problem, or work around it, while remaining compatible with existing 
implementations. 

The message format itself consists of data elements collected into groups. 
Each data element (for example, patient name) consists of two numeric fields 
forming a key, equivalent to the name of the data element This is followed by a 
length field and the element value. The key (which is fixed length) consists of the 
group number and element number (in hexadecimal). The length Held is a 32 bit 
integer which gives the length of the value field in bytes. The value field is 
therefore of variable length, the only restriction being that lengths must be an even 
number of bytes (padded if necessary). Note that this 32 bit length allows for some 
very large elements. Though questions were initially raised about this size, recent 
work is proving its value. Figure 4 shows an example of the structure of an ACR- 
NEMA element. 
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FIGURE 4: ACR-NEMA DATA ELEMENT STRUCTURE 



Groups were defined in order to collect elements in a logical fashion. This 
was not entirely successful as it seemed some elements could be located in two 
places. Since elements are not duplicated (except with specific elements in group 
0000 hex), this resulted in imperfect groupings. However, the group-element keys 
are unique. The structure also provides for expansion in that some "room" was 
reserved between assigned group and element numbers to allow for additions. To 
provide for user-defined or manufacturer specific data, a structure of shadow groups 
was defined. This also provided a mechanism by which new groups or elements 
could be used until considered for incorporation into revisions of the Standard. The 
defined groups of the Standard are given in Table 1 . 

Though it appears that the Standard contains a great many elements which 
need to be sent with each image, in fact, some elements are specific for certain types 
of images. For many medical images, the collection of elements will be a small 
fraction of the total message size. Also, the elements were assigned types based on 
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how important they are for interpretation. For example, pixel format information is 
essential whereas equipment manufacturer name is probably not The former is 
considered a mandatory data item (a type 1) and the latter is optional (type 3). There 
are also type 2 elements which are considered mandatory, but which conceivably 
may not be known at the time of the examination. Unlike type 1 elements, which 
must be sent and must contain valid data, type 2 elements may be sent with a zero 
length, indicating that the value is not known. Defaults exist for many elements 
and fall into type ID and 2D categories. The difference between these is that the 
default is always assumed if a type ID element is not sent. It is not possible to 
send a zero length (indicating "unknown* 1 ) for a type ID element whereas this is 
possible for a 2D (in which case the default is not assumed). Type 3 data elements 
are those which are not essential for interpretation, but which may be important for 
certain medical images. 



Group 
Number 



Table 1 

ACR-NEMA Standard Groups 



Type of 
Information 



Description of Group Contents 



0000 

0008 

0010 
0018 

0020 

0028 
4000 

6000-601 E 



Command 

Identifying 

Patient 
Acquisition 

Relationship 

(mage presentation 

Text 
Overlay 



Information which provides for end-to- 
end communication capability between 
two pieces of equipment 

Information which provides unique 
image identification parameters to the 
user 

Information related to the patient 

Information related to the image 
acquisition device and imaging 
procedure 

Information relating the image to the 
location within the patient and to other 
associated images 

Information related to the parameters 
of image presentation 

ASCII text 

Information related to overlay pixel 
data associated with an image 



7FE0 



Pixel data 



Image pixel data 



Once the message is assembled, it is sent down to the Session layer. The 
Standard has provided "hooks" in this layer for a multiple virtual channel protocol. 
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In effect, virtual channels allow the single physical connection to support multiple 
logical connections. Connection control words and message source and destination 
fields are defined in this layer for incorporation in layers below. 

The Transport/Network layer takes the message with the information from 
the Session layer, and fragments the message into packets. Packets consist of a 
block of the message (0 to 2048 bytes) plus a packet descriptor word and block 
sequence number. The packet descriptor word contains the information on channel 
number as well as packet type (channel control) and packet service class (ix., 
whether or not to acknowledge each packet). Packet structure is shown in Figure S. 
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FIGURE 5c ACR-NEMA PACKET STRUCTURE 



The now packetized data proceeds down to the Data link layer. Each packet is 
enclosed by a frame descriptor word and frame check sequence to form a frame. The 
frame descriptor word consists of a frame type (e.g., data and status) and a parameter 
field (length or status depending on frame type). The frame check sequence is a 
checksum which allows for error detection. Figure 6 shows the structure of an 
ACR-NEMA frame. 
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PACKET: 



FRAME CHECK SEQUENCE 



NOTE: ALL WORDS ARE 16 BITS LONG 
FIGURE 6: ACR-NEMA FRAME 
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Frames are sent to the Physical layer which handles transmission across the 
interface. The physical interface consists of 16 asynchronous parallel lines with 
control signals. The drivers for the signal lines are EIA RS48S standard, driving 
differential pairs for high noise immunity. The connector is a widely available 50- 
pin micro-type and must be female on the equipment. Since the interface is 
designed to operate in a master-to-master environment, the control lines cross over 
in the cable. With the specified drivers and a cable length of IS meters, hardware 
speeds of 8 megabytes per second can be achieved. Since the interface is 
asynchronous, however, much slower rates (limited only by the 1 second idle 
timeout) are accommodated. Figures 7 and 8 show the signal lines of the interface 
and the interface electrical connector. 
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FIGURE 7: ACR-NEMA PHYSICAL INTERFACE 
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FIGURE 8: ACR-NEMA 
PHYSICAL 50-PIN 
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On the other side of the interface, the frames would proceed back up through 
the layers, eventually resulting in a reassembled message. Figure 9 shows overall 
flow through the layers. 
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FIGURE 9: DATA FLOW THROUGH THE ACR-NEMA INTERFACE 
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While the Standard at first seems complex, in fact, a minimum for 
compatibility has been defined. This states that of the commands, only SEND need 
be implemented. Also, only a single channel need be supported. The full 
information on minimum requirements has been published by NEMA.^ 



THE OTHER STANDARDS: DATA COMPRESSION 

Following publication of ACR-NEMA 300-1985, it was recognized that 
additional work was necessary as new requirements were made known to the 
Committee. New Working Groups were formed to examine these issues. Initially, 
data compression was assigned to Working Group IV, exchange media to Working 
Group V, and validation issues to Working Group VI. 

The vast volumes of data generated in medical imaging departments put a 
strain on archival media, even optical disk. The communications systems which 
must handle the volume are also burdened. To help alleviate this problem, data 
compression has been proposed as a solution. This paper will not address the 
benefits of lossless versus lossy data compression as this was also not the task of 
WG IV. Rather, the Working Group sought to allow for data compression and 
decompression in a manner consistent with the existing Standard. 

It was recognized that data compression consists of two steps. In the first the 
data is operated on so that it can be represented in a different way. For example, 
differences between pixels might be computed so that the data may be represented by 
these differences rather than the full pixel values. The second step consists of 
coding the converted data so that less space is needed for its storage. In reviewing 
compression methods applied to monochrome medical images, three popular 
conversion methods and three coding techniques were identified. 

Supported conversion techniques are: differential pulse code modulation 
(DPCM), discrete cosine transform, and the pyramid (and its relative, the S) 
transform. DPCM is a differencing method which represents original data by a set 
of differences. The discrete cosine transform is a real valued transform for which fast 
algorithms exist. The pyramid and S transform are also differencing methods but 
compute sets of differences (in horizontal, vertical, and diagonal directions) from 
pixel data. The pyramid and S transforms have the useful property of producing 
viewable images, reduced in resolution, at each step. 

Once converted, the coding methods take advantage of redundancy in the data 
to do the compression. The compression standard supports run length, Huffman, 
and Lempel-Ziv methods. Depending on how the data are quantized, the 
compression may be bit-preserving (lossless) or non bit-preserving (lossy). An 
example is in quantizing the cosine transformed data. If the quantization extends out 
until transform values vanish, there will be no high-frequency information lost. 
Truncating the transform data before this point will cut off such high-frequency 
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information. Loss of the high frequencies corresponds generally to a decrease in 
edge sharpness and diminishing of fine detail. 

Another popular compression method which is lossless but docs not fit well 
into the above list of methods is perimeter encoding. In this technique, which is 
used for nonrectangular images, the zero values outside the actual image area arc not 
saved. A map describing the number of pixels per line is sent instead. This method 
is also supported by the compression standard. 

The design of this standard is such that methods may be combined if such 
combination will yield more compression. The compression standard consists of 
new groups and elements which allow a device to describe how compression was 
done. For proprietary algorithms, the standard provides a structure for a 
manufacturer to send these data about the compression used. The structure of the 
compression standard was designed to be flexible and allow for future growth. 

The compression standard was voted on by members of the ACR and NEMA 
and was approved. The document is currently available from NEMA as ACR- 
NEMA Standard PS -2 at nominal cost. 



EXCHANGE MEDIA 

Working Group V of the ACR-NEMA Committee was charged with 
evaluating exchange media for possible standardization. When the Working Group 
was formed, the most popular exchange medium was one-half inch magnetic tape. 
Though there was considerable interest in optical disk, there was no industry 
standard for it. The interest in archival as well as exchange media came from the 
manufacturers who preferred to write one set of software for generating exchange or 
archive volumes. 

Since there was no consensus on optical media standards at the time WG V 
began working, the group decided to consider magnetic tape first Since the charter 
is to consider exchange media in general, as industry moves towards standards for 
optical media (for example, the 5.25-inch medium) the Working Group may 
continue to evolve standards. 

As the full ACR-NEMA Committee had done, existing standards were first 
reviewed. The AAPM Magnetic Tape Format « was a very good possibility, but 
did not address all the issues required for exchange. The manufacturers thought that 
too much modification of existing operating system drivers would have to be done. 
LANDS AT, NATO, and Japanese JAM IT were among the other standards reviewed. 
None was thought to be as good as the AAPM standard for medical use. The 
Japanese standard, in particular, had fixed-length fields. If possible, the desire was 
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simply to write out an ACR-NEMA data set onto the exchange medium. While 
manufacturers had widely differing formats for data, a common element proved to be 
use of the ANSI standards for magnetic tape. These covered both the physical media 
standardization and common labels and file structures. Using the ANSI standards, it 
was possible to fit the ACR-NEMA data set structure into the ANSI standard file 
structure with a "wrapper" consisting of ANSI labels. 

As of this writing, the draft of the magnetic tape standard has been balloted, 
but one negative vote required ACR-NEMA to address the associated comments. 
This has been done, and the standard is being re-balloted. The Japanese PACS 
standards organization has approached ACR-NEMA WG V with a proposal for an 
exchange standard based on 5.25-inch erasable optical disk. This proposal is 
currently undergoing evaluation, and it does seem to have a fairly wide industry 
backing. 



VALIDATION 

Working Group VI was formed to examine issues related to validation of the 
"parent" standard. This includes both validation of the Standard itself, and assistance 
of those implementing or testing hardware and software for the Standard. The 
Working Group serves as the clearinghouse for questions about the Standard and for 
suggestions regarding new data groups or elements. This group also examines 
issues related to revision of the Standard, and defines how revisions will be 
compatible with prior versions. The basic guideline has been that no element would 
be changed or replaced unless found to contain errors. 

The first revision of the Standard (version 2.0) has passed the vote of the 
ACR and NEMA members, and is now available from NEMA as ACR-NEMA 300- 
1988. This revision includes new elements for definition of an image identification 
hierarchy, elements added for ultrasound, and commands and elements added to 
support laser film writers and other hardcopy devices. All changes are compatible 
with version 1.0 of the Standard. 

In addition to the Version 2.0 of the Standard, WG VI has published a 
number of engineering bulletins, errata lists, and responses to questions pertaining 
to the Standards. The group has sponsored a special meeting for implementers, and 
has published the results of that meeting. As of this writing, plans have been 
finalized for a hardware workshop, during which manufacturers will connect their 
ACR-NEMA test hardware together. The results are expected to provide a great deal 
of information about how well different implementations can work together, and 
give direction to further efforts of WG VI in assisting interconnection. 

This Working Group has been the busiest, and its work continues. As 
discussed in a later section, the revisions for version 3.0 are presently being studied. 
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Two additional Working Groups have been formed to examine other issues 
which have been brought to the attention of the Committee. Multidimensional data 
was assigned to WG VII, and interfaces between hospital information systems 
(HIS), radiology information systems (RIS), and PACS are being developed by WG 
Vm. The original three Working Groups have been deactivated, though many of the 
members are now active in the newer WGs. 



MULTIDIMENSIONAL DATA 

With the increasing interest in images which are multidimensional, either in 
spatial dimensions or with parametric features, Working Group VII was formed to 
examine how such data could be accommodated in ACR-NEMA 300-1985. The 
Standard currently specifies that images are rectangular with the number of 
dimensions allowed to be "2". 

This Working Group has some working proposals for member use, but has 
suffered from difficulty assembling all members for meetings. Some of the newer 
work of WG VI may supersede the activity of this group. The Working Group is 
on inactive status, but can be reactivated when needed, and is nonetheless seeking 
suggestions from interested users or manufacturers. 



HIS/RIS/PACS INTERFACES 

The newest of the Working Groups is defining a standard for interface of HIS, 
RIS, and PACS. Unlike the parent Standard, the product of WG VIII will be an 
upper level standard. It is thought unlikely that, in the heterogeneous computing 
environment formed by these systems, the hardware parts of the Standard would find 
much acceptance. It will most likely build around the parent Standard so that 
ACR-NEMA messages (or parts) may cross from HIS to RIS to PACS and back. 

The necessity of this effort is based on the requirement of having certain RIS- 
or HIS-based data in the PACS (patient demographic information, for example), and 
getting PACS data (such as time of examination) back to the RIS or HIS. 
Allowing direct exchange reduces the problem of operator errors in transferring 
information between systems. Currently, much patient information has to be re- 
entered in the PACS. Errors lead to database inconsistency which can result in an 
inability to access data and in duplication of files. A similar situation pertains to 
PACS data which must be copied into the HIS or RIS. 

The WG began by defining an entity-relationship model of radiology in order 
to understand what kinds of data would exist at different places in a radiology 
department and what the relationships between entities (such as "patient" and 
"report") are. This effort resulted in developing the necessary new data set groups 
and elements, the groups being thought of as the entities and the elements as the 
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attributes of the entity. The current effort is to develop a functional or operational 
model of radiology so that the necessary commands can be similarly defined. Work 
in these areas is being done with careful attention to the standards work of the IEEE 
Medix group as well as the HL7 and HISCC. These groups are working on 
standards which will affect the HIS and RIS sides of the interface. 



FUTURE DIRECTIONS 

The original ACR-NEMA Standard has undergone one revision, and has 
spawned two new standards with a third on the way. All of these have retained the 
structure of the parent Standard, and are compatible with it in various ways. 
However, as user requirements mature in PACS environments, even more changes 
to the Standard arc anticipated. 

The most ambitious effort at present is directed by Working Group VI in 
conjunction with Working Group VIII, and consists of examining proposals to 
define a version of the Standard which would operate more easily over networks. A 
joint development by Philips and Siemens, the Standard Product Interconnect (SPI) 
is one model being used. Another is based strongly on the work of WG VIII, and 
addresses a combination of the application level work being done by that group 
along with the changes necessary in the lower layers so as to support network 
interconnection. A definite possibility for the future is to "split" the Standard. The 
upper layers would be made ISO-OSI compatible, or at least more ISO-OSI-like, 
while retaining operability with the lower layers. The result would be a standard 
which could run over the existing hardware interface of Versions 1.0 and 2.0, but for 
applications requiring network operation, or connection between networks, could use 
other hardware. 

This also relates to a frequently asked question about availability of ACR- 
NEMA implementations. As of this writing, implementations for VME bus to 
ACR-NEMA, IBM PC/AT bus to ACR-NEMA, and Digital Equipment 
Corporation Q bus to ACR-NEMA exist, though some are admittedly prototypes. 
The Japanese manufacturers have indicated that they are working on both full ACR- 
NEMA implementations as well as a minimum requirement version they call 
MIPS. Sanctioning the use of the ACR-NEMA data structure over non-ACR- 
NEMA interfaces creates a problem in that it has the potential to result in a 
multitude of hardware-incompatible devices. However, many of the ACR-NEMA 
Committee members see this change as a necessity as it will allow the ACR- 
NEMA data structure to be more widely used. The hardware interface will still be 
available, and for applications which require its speed, is still highly desirable. 
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THE USES OF THE ACR-NEMA STANDARDS 

The primary uses of the ACR-NEMA Standards are dependent on what 
functions are being supported. The 300-1985 and 300-1988 interface standards arc 
designed primarily to connect image generating equipment lo other devices. Those 
other devices may include network interface units (NIU), teleradiology transmitters, 
laser film writers, diagnostic workstations, or even another piece of imaging 
equipment. In a PACS, the most likely connection is to an NTU or data acquisition 
device (AD). In this instance, the ACR-NEMA interface is on one side of the NIU 
or AD and the PACS network connection on the other. If the network does not 
support ACR-NEMA data structure directly, the NIU or AD then serves as a 
protocol converter. 

A frequently raised point about the interface Standard is that its performance 
dictates significant processing power, and that unless a low cost version is available, 
it is unlikely to be used on equipment which costs only four to five times what the 
interface does. This is a valid criticism, and can be addressed in two ways. The first 
is that low-cost interface hardware is becoming available. The PC/AT card version 
is a good example. This would allow construction of an NIU or AD based on a PC 
rather than a minicomputer. 

The second solution involves architecture of the PACS. While it may not be 
cost-effective to use a separate ACR-NEMA interface on each piece of equipment, it 
may be economical to use a single interface with multiple devices connected to it. 
Examples arc in ultrasound and nuclear medicine. In the former, multiple video 
outputs (or digital, if available) could be connected to a central computer which 
would manage the data from the machines so as to keep the images sorted by case. 
This computer would have a single ACR-NEMA interface for PACS connection. 
Nuclear medicine would be similar, and is more likely to evolve this way since 
many nuclear medicine departments are already networked to a central computer 
system. These approaches are not very different from systems which currently use a 
shared laser printer with a computer front end to keep the images sorted out 

The data compression standard will be of value in reducing PACS storage and 
network bandwidth requirements. Even a lossless compression of 2:1 doubles 
archive capacity. The use of the lossy compression techniques requires that those 
who will be using the expanded images accept this limitation. Since the techniques 
are often computationally intensive, lo be of value for reducing network bandwidth 
some form of hardware for fast compression and expansion will be required. 
Research on lossy compression and hardware for performing the algorithms is 
presently underway. It may be possible to use compression selectively, 
compressing images only after they have been interpreted or only if they are being 
sent to non-diagnostic workstations. 
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The exchange medium standard, which currently supports magnetic tape, will 
find use in inter- institutional exchange. Though engineers have been predicting the 
demise of magnetic tape for some time, it manages to remain heavily used, even if 
not very popular. The emergence of a standard for optical disks, particularly the 
5.25-inch format, will eventually result in phasing out magnetic tape for exchange. 
The 12-inch optical medium has, unfortunately, progressed quite far with no widely 
accepted standard, so that standardization now is unlikely, Industry has had more 
success in developing support for the smaller medium. Another potential use of the 
exchange medium standard is for use in a per-patient or personal health data system. 
In these proposed systems, each patient's medical record information is written on a 
small, removable medium (probably optical). The idea is to allow the patient to 
carry his or her data around to different healthcare facilities as needed. The necessity 
of a standard for this application is self-evident 

Communication between the different information systems in a medical 
facility could benefit from use of the standard which is being developed by WG VIII. 
The HIS/RIS/PACS standard will of necessity be an upper-layer one. It will, along 
with standards efforts in the HIS and RIS communities, help promote the concept of 
an electronic medical record. This idea continues to grow in popularity as the 
advantages of a record which is amenable to database operations become more clear. 
As healthcare becomes more regulated, the production of reports demonstrating 
efficacy or documenting the necessity of treatment will be an increasing burden if 
paper records have to be searched manually to retrieve the necessary information. 



CONCLUSION 

In the seven years since its formation, the ACR-NEMA Committee has 
succeeded in producing and interface standard which is beginning to be used in a 
number of PACS. This standard has undergone one revision, and changes for the 
next version are being studied now. In addition, they have written a standards for 
image data compression and exchange media, and are working on a standard for 
HIS/RIS/PACS interfaces. These efforts are guided by the radiologic community as 
well as by representatives of industry. The collaboration between those who 
establish requirements and those who must build equipment to meet them has been 
very productive. 

The ACR-NEMA Committee welcomes comments on all of its efforts. In 
particular, WG VI is interested in hearing from those who are building ACR-NEMA 
interfaces and WG VIII in those who are working on other standards for healthcare 
information management The various Working Groups can be contacted through: 

National Electrical Manufacturers Association 
2101 L Street, NW Suite 300 
Washington, DC 20037 
Attention: Staff Executive, MedPACS Section 
Phone: (202) 457-1965 
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Image storage and image access are important aspects 
of PACS and deserve much attention by the system de- 
signer. The reporting radiologist has very fast access to 
a bank of images and has seemingly already optimized the 
process of information transfer: visual input and verbal 
output. His expectations of PACS were expressed a decade 
ago as faster and more reliable access to the diagnostic 
image bank. In addition such access should be provided 
from various locations. Technical implementations, 
though, are still falling short of meeting such 
expectations. 

Optical storage technology promised a decade ago to 
offer high density digital image storage of archival 
quality at low cost. [1] . We find now that magnetic data 
recording technology has made more progress than optical 
technology and offers fastest data rates at lowest costs 
for removable media- The isolated storage device, 
however, does not determine the effective image access 
time but the overal organization. In order to better 
define the task a typical workload of a PACS radiology 
department will be assumed. 

The interrelationship between resolution, storage 
capacity, access time and cost can be seen in Fig 1, 
which cites AHA data [2] and lists the number of 
procedures per bed and year for larger US hospitals. For 
plain film radiography we find that 124 procedures are 
performed per bed and year. The next columns show Images 
per bed and year, square feet of film, images per 
procedure, data -volume per image (kB/I) and MB/b/y. 
According to these projections a 500 bed hospital would 
produce about 3500 MBs of image data per day. 
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SYSTEM ARCHITECTURE 

While it is generally assumed that PACS is a 
distributed system, several implementations are possible 
and have been described [ 3, 4 ]. A system architecture 
which resembles that of a conventional department with a 
film library and a film archive is shown in Fig 2. It is 
topological ly a "star 11 configuration with the "Digital 
Image Library" at the hub and input devices ( modalities) 
as well as output devices (workstations) at the rim. 
This, incidentally, is the topology of the "CommView" 
PACS, developed by AT&T and marketed by Philips. 

Data quantities as listed in Fig 1 are assumed to be 
supplied by the respective modalities and converge 
ultimately on the Image Library. The conversions of the 
original modality image format into a compatible format 
will not be discussed here, but it is assumed that such 
compatibitly exists and also that no data reduction 
algorithm ( such as compression ) is applied. Two types 
of transfer paths are assumed : a direct path, where 
images and all relevant patient information are 
transferrred directly into the Image Library and a second 
path where images are transferred into a workstation. 
Throughout this paper the emphasis is on images while 
generation, transfer and storage of patient data is 
assumed to implemented in a satisfactory manner. 

It is furthermore assumed, that the primary diagnosis 
is established at the modality in question. This 
assumption may not be acceptable to some promoters of 
PACS but clarifies the underlying concept of PACS. For 
CT, MRI and US images this assumption poses no challenge. 
For radiography, however, as well as R & F and DSA images 
PACS is generally assumed to offer advantages also for 
the diagnostic process. In particular Teleradiology is 
only meaningful if it can handle traffic of diagnostic 
images and not only "review images". Still, this 
paper deals only with the organization, traffic and 
storage of images for which the primary diagnosis has 
already been established. Target of optimization is 
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access to archived images by medical specialties as well 
as authorized persons outside the hospital, including 
referring physicians. 

Images have to be transferred into the PACS 
environment with minimal disruption of the revenue 
generating operation of the modalities. This means that 
the data transfer rates into the PACS environment must 
match the average acquisition rates and may even have to 
match the maximum acquisition rates. It is a further 
assumption that image data are not transferred back into 
the generating modalities. This assumption is 
increasingly endorsed by experts who recognize that an 
omnipotent image processing workstation is unrealistic. 

The traffic out of the Image Library into 
workstations is bidirectional because edited or 
postprocessed images will also be deposited in the Image 
Library. These workstations are information centers for 
authorized PACS participants and supply images related to 
diagnostic cases as an important type of information. 

There is no direct connection between workstations, 
but only via the central Image Library. This arrangement 
may again run counter to some desires but is intended to 
preserve the authority of the reporting modality experts. 
Only if and when such experts have released images for 
which they are responsible can others gain access via the 
Image Library. This restriction is similar to good 
practices in a department. 

The Optical Jukebox is the long term archive and is 
attached to the Image Library. It will hold image data of 
nonactive cases on removable media. As mentioned above 
the data base controlling search and retrieval is not 
part of this discussion, but it should be assumed that it 
exists and performs satisfactorily. 

INPUT IMAGE DATA RATES 

Both data rate and data quantity vary from modality 
to modality. Digital cine for cardiac examinations 
produces a data rate of a least 8 MB/s. This rate is 
equivalent to 30 images of 512 by 512 resolution and 8 
bits data depth. ( Fig 3 ). This data rate can be stored 
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on fast parallel disks but an examination may fill a 
large disk with the result that the next examination has 
to be delayed until the data have been transferred into a 
permanent storage. This second storage has often a much 
lower data rate. Although optical technology development 
was expected to produce equipment which can offer a data 
rate of 50 Mb/s (6.3 MB/s ) [ 5 ] f such development has 
not yet occured. Magnetic removable storage, has been 
demonstrated which can handle such data rates [6, 7]. 

Optical tape [ 8 ] f as recently announced, offers a 
sustained data rate of 3 MB/s, but even this is too slow 
for on-line digital cine recording. 

CR ( Computed Radiography ) is producing a large 
amount of image data but at a rate of only 83 kB/s. The 
limitation is the readout of the storage phosphor which 
is determined by a physical time constant. It is 
unlikely, that the readout time can be significantly 
shortened. 

Ultrasound in dynamic mode supplies data at cine 
rates but with less depth than cine angiography. 

Conventional video recording suffices for on-line storage 
prior to data editing. 

Nuclear cardiology also produces cine data but of 
limited resolution so that magneto-optical recording or 
even WORM recording would suffice. 

CT and MRI are already offered with optical WORM 
drives and pose no challenge. 

COMPARISON OF REMOVABLE RECORDING MEDIA 

Fig 4 list some of the most significant media for 
medical diagnostic recording. The diskette ( floppy ) is 
still used but has limited capacity, although new 
developments will increase both capacity and data rate 
[9 ]. Removable magnetic disks are relatively expensive 
and of limited capacity and data rate. Magnetic tape is 
listed as reliable medium, certified for archiving of 
important information. Helical tape, capable of recording 
at high data rates is offered by two companies: 
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ARCHIVING OPTIONS 

500 bed hospital, 5 GB/d, 363 procedures/d, 
121 patients/d 

option 1:every patient has a 500 MB disk 
121 disks must be loaded daily 
30 f 250 disks per year 



option 2:Jukebox with 100 disk, 2 GBs each 
2.5 disks/d to be loaded 
625 disks per year 

option 3: Terabyte optical tape system 
5 GBs need 5 m of tape 
1.25 reels per year 

Fig 5 

Honeywell has some time ago announced the VLDS and 
Sony has recently shown a "Digital Instrument Recorder 11 
with a recording rate of up to 32 MB/s. This rate would 
suffice for recording 1000 by 1000 pixel images at video 
rates. 

Optical disks of the WORM type are still slow ( .25 
MB/s ) [ 10 ], but the optical tape system manufactured 
by Creo offers a sustained data rate of 3MB/s. 



ARCHIVING OPTIONS 

Short term archiving of images of active cases is 
performed by the Image Library. The average length of 
stay in USA acute care hospitals is 6 days which means 
that the Image Library should be large enough to hold 30 
GBs of image data. If access to old image must be offered 
some of the images archived in the Jukebox will have to 
be transferred into the Image Library. A first option is 
reduction of the digital radiographic cases by using CR 
mainly for ICU and emergency cases. A CR system can 
handle maximally 480 films per day (120,000 per year ) 
but realistically only a half of this can be processed so 
that the daily RAD data quantity is reduced from 3500 to 
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1200 MBs. The required Image Library capacy is then 22 
GBs. Even this magnitude is too large for magnetic disk 
storage so that access to the Jukebox will be required 
even for active cases. 

Optical Jukeboxes offer storage capacity up to a 
Terabyte (1000 Gbs ) and access time of the order of 6 - 
10 sec. The optical tape system by Creo has slower access 
time but offers higher data rate and reduced cost. There 
is renewed interest in distributing image archiving over 
many disks whereby every patient would have an optical 
disk assigned. Fig 5 lists three options for a "filmless" 
500 bed hospital. The daily procedure load of 363 is 
represented on the average by 121 patients. For a full 
year this amounts to 30,250 disks which have to be 
stored, accessed and updated. 

Option 2 would perform long term archiving on a 
Jukebox with 100 disks each of 2 GBs capacity. 625 disk 
would be needed per year or 2.5 disks average per day. 

Option 3 uses the Terabyte optical tape system which 
can stored the yearly workload of 1,250 GBs on 1.25 reels 
of tape. The daily data quantity would only take up 5 
meters of tape. 

Obviously there are advantages and disadvantages with 
any one of these three options, but technology has 
progressed to the point that such options are available 
and can be considered for a realistic implementation of 
PACS. 

CONCLUSION 

If a primary requirement of a useful PAC system is 
fast access to a large number of images organization of 
image storage has be optimized. This will require 
assessment of the various magnetic and optical storage 
systems but may also require a novel approach to storage 
space allocation. Fast magnetic disk offer already fast 
access to large data blocks. A review of the storage 
requirements of a "filmless" radiology department leads 
to the conclusion that short term archiving all of the 
radiographic images would become complex and very 
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expensive. The suggested solution is optimized design of 
the long term archive for fast access and reduction of 
the digital radiographic workload. 
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Introduction 

If the MGH Q-RSTAR PACS is to be viable as a clinical system, it ultimately, 
will archive approximately 4-5 million megabytes of image data per year, yet be 
called upon to provide very quick storage-to-screen access times for individual 
images, each typically containing 8-16 megabytes of data. In designing Q- 
RSTAR, it became clear that any discussions about the image storage system 
without full consideration of the communication methods and algorithms was 
impractical. This paper will describe four areas of development within the Q- 
RSTAR system. They are: statistical image caching, distributed image storage 
and management, local area networks, and wide area networks. 

Statistical Image Caching 

It is well known that, in general, cost for a particular type storage device 
increases dramatically with decreasing access time and increasing throughput. 
An archive consisting solely of solid-state memory may have exceptional 
performance but suffers from astronomical cost Conversely, an archive based 
upon ultra-high density helical scan magnetic tape drives has a low cost per byte 
but suffers from very poor performance. 

Given the semipredictable pattern of image retrieval within the hospital 
environment and the obvious relationship between storage cost and performance, 
the following question becomes conspicuous: Can the image archive be 
constructed so that images are moved from slow, inexpensive storage to higher 
performance storage in anticipation of their actual use, as a means of obtaining 
high performance at an affordable cost? 
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Figure 1 : Caching and Preload 

Specifically, can image data be moved from slow, inexpensive storage based on 
need or anticipated need to high performance storage elements? Conversely, can 
image data be purged from the high performance elements based upon space-need 
or inactivity? Figure 1 offers a schematic representation of this concept. In the 
Q-RSTAR system, the most inexpensive storage devices are the removable 
media (for example optical platters and magnetic tapes) that have been placed 
off-line. Moving right in the diagram, mounted volumes are clearly more 
accessible but occupy resources (for example an optical drive or slots in a 
Jukebox). In the Q-RSTAR system, a device named the 'Cluster-support' unit 
contains, among other devices, a 3.4 gigabyte, high-performance, parallel 
transfer magnetic-disk-array. This magnetic-disk-cache is intended to quickly 
service requests from workstations needing image data. Additionally, within a 
Q-RSTAR workstation is a small (4 to 16 megabytes) solid-state based image 
buffer (or cache) allowing repeated requests for a small subset of images to be 
serviced quickly and locally. 



The goal is to move images along this hierarchy so that the images arrive at the 
high performance elements just as the user needs them. The key to achieving 
this goal is in implementing a set of algorithms that anticipate where and when 
images may be requested. 

In the process of rendering patient care, many events take place that indicate the 
future use of radiological images. Scheduling a patient for radiological exams 
indicates a strong probability for prior exams to be recalled. Obviously, the 
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completion of an exam indicates that these images will be accessed soon. 
Viewing the first image of a CT series is an indication that the others soon will 
be requested The Q-RSTAR system uses these techniques to manage its image 
archive and receives its primary caching information from the Department's 
Radiological Information System, DecRad (a Digital Equipment Corporation 
product). 

Distributed Image Storage 

Since Q-RSTAR will service many separate facilities located throughout the 
Boston metropolitan area (e.g., Boston, Charlestown, Somerville, and 
Cambridge), the system supports a distributed image database. For reliability, it 
is useful to maintain copies of currently active images within more than one 
archive unit Again, this necessitates the need to maintain a distributed database. 

The basic "raw materials'* needed to implement this distributed image storage 
scheme are Storage-units. A storage-unit is simply a device that is capable of 
storing strings of bytes representing images. Storage devices can be graded 
according to several criteria: (1) cost/megabyte of capacity; (2) typical access 
time to store or retrieve images of various sizes; (3) reliability of media; (4) 
reliability of device (e.g. MTBF); (5) the communication protocols through 
which they are able to send and receive images. Q-RSTAR is able to integrate a 
wide spectrum of storage devices with different configurations of these 
parameters. 
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Figure 2: Distributed Caching 



The Q-RSTAR distributed image database provides a framework for integrating 
all available Storage-units into a single distributed image database and cache 
regardless of where the Storage-unit is located. Figure 2 shows the structure of 
the distributed image database. In the figure Storage-units A and B may reside 
on a VAX computer, Storage-unit C may be a parallel-disk-farm hosted off of a 
Macintosh, and Storage-unit D, a buffer residing in a user's workstation. 

As shown, each Storage-unit's communication channel is "wrapped" in an 
adapter which gives it a common interface to higher level software. A common 
layer of software called the "Shuffler" arranges transfers of individual images 
between different storage facilities using available communication channels, 
while a still higher layer, called the "Executive" implements the caching strategy 
and image management functions. 

Local Area Networks 

For this caching strategy to work, the bandwidth between various Q-RSTAR 
components, in particular between the magnetic-disk-cache located in the 
Cluster-Support unit and the workstations, should be high. We considered a 
number of LAN'S such as FDDI (Fiber Distributed Data Interface) and Ultranet 
(Ultranetwork Technologies Inc.), to name a few, but they offered a poor price- 
performance ratio in relation to the number of units needed for the deployment 
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schedule planned for 1990 and 1991. As described in other papers, the intra- 
institution communication substrate for Q-RSTAR is based upon the concept of 
a Dual LAN as shown in figure 3. An industry standard, Ethernet, is used to 
carry command & control information between the Cluster-Support unit and the 
workstations. A high performance, but functionally primitive, optical LAN, 
based upon Advance Micro Devices Inc. TAXI chips, carries image data. This 
approach has met the required price-performance goals. 



To 
Head-End 
6.7 MB/sec 




Optical LAN 
12 MB/sec 
Image Data 



Ethernet 
0.2 MB/sec 
Command 
& 

Control 



To 
Work- 
Stations 



Figure 3 : Dual LAN 



Wide Area Networks 

The MGH department of Radiology is itself geographically distributed and serves 
a number of geographically distant affiliated institutions. To meet the needs of 
high speed data interchange between the various Q-RSTAR system locations 
both 45 megabit/second DS3 and 1.5 megabit/second Tl circuits have been run 
to the locations as shown in Figure 4. Bridges and gateways interface the LAN 
and WAN (wide area network). 
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Figure 4: Wide Area Network 



Overall System Interaction 

The primary goal is to create a system that provides for image availability at a 
reasonable cost but with high performance and throughput in a wide-area 
distributed computer environment. 

In this system all archives (long or short term) and intermediate storage 
elements, whether they be disks, tapes (both on-line and off), or computer 
memory buffers are treated as Storage-units. A Storage-unit presents a common 
interface regardless of the media or technology. Connecting all Storage-units is 
a directed graph of communication 'pipes' provided by either the LAN'S or 
WAN'S managed by the system's Shuffler. The Shuffler's responsibility is to 
manage all pipes and to orchestrate the orderly transfer of data from one Storage- 
unit to another. At the top of this pyramid is the distributed image management 
system's Executive. The Executive uses heuristics about 'how' the department 
operates to service requests for image data availability in an anticipatory manner. 



438 



Additionally, the Executive manages the redundant placement of active images 
on a multiplicity of Storage-units. 

Many systems have LAN'S, WAN'S, distributed storage, and caching strategies. 
It is felt that the unique contribution of Q-RSTAR is its effort to integrate all 
available storage facilities and communication resources into a unified 
framework. 
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Abstract : 

in the domain of radiological image compression it is known that the 
resulting distorsion due to this transformation is a very critical point. 
But it is also known that today with the actual applied technologies in PACS 
it is nearly impossible to store and transmit in an efficient way this huge 
amount of medical imaging data. Compression is therefore an essential mean 
of reducing this large storage requirement and the related archiving and 
transmission charges. 

A new adaptive image oriented methodologyis presented i nth is paper for 
radiological image compression which consists in characterization and 
classification of radiological images leading to an optimal choice of encoding 
algorithm for the various image classes. 

The classification in itself is based on textural features and uses a Karhunen 

- Loeve expansion and discrimination which provide very interesting 
results 

Because of the large number of image classes present in radiology we 
developed a method called M discriminating pyramid " which consists in a 
succession of discriminant classifications, and which ends in obtaining a 
well-defined image class with the appropriate encoding algorithm. 

1. Introduction 

Quite recently PACS (Picture Archiving and Communication System) [l]has 
become more and more a reality in a modern hospital. 

The justification of such an image management system is based on different 
factors: 

- more efficient diagnose possibilities( quicker retrieval of older archived 
images, possibility of postprocessing of numerical images, ...) 

- less films are produced, a more rational way of management in a 
radiology department 
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- efficient image communication between an in geographical separated units 
distributed radiology department 

But this new concept involves the transmission and the storage of huge 
amount of data, in a world where the number of radiological image producer 
and related examinations is still increasing. The spatial resolution is also 
growing in an important way, specially in the field of conventional digital 
radiology ( about 2048 x 2048 pixels => about 4 Mbytes/image ). 
Even if the technologies related to this recent concept are going to be more 
and more able to support such a technological challenge, there still are 
problems to fulfill an efficient management and manipulation of such 
amounts of data. 

And by this reason image compression is a very important component of a 
PACS [1][15]. 

But specially in medical images, when using efficient irreversible 
compression methods, distorsion is a very critical point of acknowledging 
whether yes or not a method is applicable to a special kind of medical image 
[15]. And there are no adequate objective or analytical methods yet of 
evaluating distorsion , so it is often preferable to use ROC studies to have a 
subjective criteria of the performance of an algorithm. 
Also compression must be for the user an invisble task demanding of him 
less as possible interaction. 

In this paper we give first a global overview of different kinds of 
compression methods. In a second chapter we give a new approach of 
classification of medical images in function of their textural content and 
then relating the most efficient encoding algorithm an its compression ratio 
to this class of image. 

In a third chapter we give an example of context sensitive segmentation 
permitting to encode most effeciently the different kinds of information 
types present in an image ( background , anatomic region , text ) 

2. Qiptai Qvervtew pf lmege compression Algorithm? 

During the last 20 years one could observe a large interest in research of 
developing new methodologies for image compression [2] applied in fields as 
different as aerospatial imaging, industrial image processing, video imaging 
and finally medical imaging. 

Generally image compression methods are subdivided into two classes: 

- exact or error-free encoding 

- irreversible encoding 
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Exact or error-free encoder are often very efficient for non-natural 
images as graphics or synthetic images. For natural images like medical 
images they give compression ratia about 2 to 3. 

More efficient are irreversible compression algorithms, permitting in 
function of acceptable distorsion to obtain compression ratia of about 10 to 
15. 

But there is an important difference .that is the fact that irreversible 
compression algorithms are more complex and so need for time acceptable 
compression a more sophisticated hardware architecture. 

2.1 Error-free or Exact Encoding 

We give in this chapter the principles of the often used error-free 
compression algorithms. 

The main advantage of these methods is that they are easily implementable 
and that they dont be based on a complex theory. 

Therefore this algorithms can even run on not sophisticated architectures, 
but in medical image compression when used in a direct form they don't give 
important compression ratio (2. ..3). 

° Huffman Coding [4] 

The Huffman coding (entropy coder) uses the fact that in a grey-level image 
the probability distribution of grey levels is normaly not uniform, and so 
this method attributes shorter codes to those levels, which have the most 
important probability of occurrence in the image and longer codes to those 
levels having less important occurrence probability. This implies that one 
obtains variable length codes and so a neccessary condition is to respect the 
prefix rule, giving the possibility to separate concatenated codes 
afterwards on the receiver side. For an eight-bits level image the most 
important compression ratio one could obtain is 1 bit/pixel (uniform grey- 
level image ). 

° Run Length Coding [27] 

Run length coding uses the fact that if successions of same valued 
pixels are present in the image, they can be coded by a pair of values, 
where the first indicates the grey level and the second the occurrence 
or frequency of this. 

If a certain threshold value of a run is not present a special flag 
indicates that the following n words are not encoded. 
The image can be searched or scanned by different ways, like row by 
row or column by column . 
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The decoder then makes a restauration of the different runs which were 
present in the original image. 

Different kinds of preprocessing operations of the image give more 
efficient run-length coding , as shifting or substraction operations. 
There exists an other family of encoders based closely on run-length 
coding, but looking for 2JD runs. 
This method is commonly called quadtree [28] coding. 

° Predictive Coding [2] 

The aim of this method is to make as well as possible a prediction a(ij) 
of the encoding pixel a(ij) and then to store with less bits as possible 
the prediction error e(i,j) = a(i,j) - a(i,j) instead of the normal pixel 
value. 

The predictor itself can be a linear combinaison of the neighboring 
pixels or a more complex adaptative predictor. 

There exists a lot of variants of how designing an optimal predictor 
(ex.: maximum likehood predictor, minimum mean square predictor,..). 

2.2 Irreversible Coding 

Irreversible coding represents a more efficient coding family 
permitting to get more important compression rates, but introducing a 
distorsion error in the decoded image. 

Irreversible coding is done either in the spatial, frequency or in a 
spatial-frequency domain [2]. 

The goal is to eliminate the present correlation in the original image 
space [3] by a transformation and thus obtaining a more compact 
representation of the image in a new space, where then the critical 
point is to quantize as well as possible the resulting parameters or 
coefficients either in by scalar or vector method. 

Most of these methods use an error-free coder after quantization of 
coefficients, for optimizing the final storage of them. This operation is 
commonly called hybrid coding. 

Generally, to evaluate the performance or the quality of the coding 
operation one uses different features like: 

- Mean Squat? grrpr ; 

MSE = Z I [(Po(i,j) - Pc(ij)) 2] (1) 
Po original image, Pc coded image 
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- Signal lo Noise Ratio : 



SNR = 20 log 10 (E[x(i,j)] 2 /MSE) 



(2) 



In literature one also can find more sophisticated measure features like 
the WMSE (weighted mean square error) [10] which tries to ponderate 
the coefficients in function of an eye sensibility criteria. 
But often these objective features don't give satisfaction in evaluating 
the performance of a compression method with its related compression 
ratio , and so it is often more useful to apply ROC studies ( Receiver 
Operation Characteristics) to evaluate the performance by a subjective 
method . 

We will describe here in a very short way different irreversible 
algorithms like: 

- Cosine Transform 

- Bloc Truncation 

- Vector Quantization 
- Subband Coding 



The cosine transform represents today the most used irreversible coder 
for several reasons: 

- the long experience and the good characteristics of DCT this implies 
that a lot of optimizing steps repose on this method (very proach to 
the Karhunen-Locve transform) 

- it is on the DCT algorithm that the choice of the ISO/CC1TT group was 
made for standardization of still image compression 

- a lot of hardware implementation exists today on the market 
permitting to fulfill DCT compression in a very rapid way 

The two dimensional cosine transform and its inverse can be written in 
the following way: 



0 



Discrete Cosine Transform (DCT) [3] 



F(u,v) = 4/N 2 c(u) c(v) £ I x(m,n) cos[ % (2m + l)u/2N] 
cos[7t(2n + l)v / 2N] 



(2) 



444 



x(m,n) = E S c(u) c(v) F(u,v) cos[7C(2m + l)u / 2N] 

" v cos[7c(2n + l)v /2N] (3) 

where c(u), c(v) = if u,v = 0 

= 1 if u,v = 1...N-1 

The computation of the cosine transform is normaly based on a FFT 
calculation [5], first applied on the rows and then on the columns, 
which permits to obtain acceptable computation time. 
The cosine transform is either applied on the whole image size (Full- 
frame DCT)[1] or on a blocks subdivided image [6] of for example [8x8] 
submatrices. 

Most quantification methods pass by an optimized adaptative [6] bit 

allocation approach, either making scalar or vector quantization of the 

resulting DCT coefficients ( see Vector Quantification). 

Of interest are one-pass methods (ADCT of the ISO/CCITT group) [7] 

because of the optimization of computation duration and hardware 

requirements. 

0 Bloc Truncation Coding [2] 

The aim of the bloc truncation algorithm is to code subimages of an 
[4x4] array size for example in a way to conserve two statistical 
moments (ex.: mean and variance) and by that way to make a local 
texture modelisation for compression purposes. 

In a first step the mean m and the variance v of the bloc are computed: 

m = 1/N 2 £ J p(ij) (4) 

v = 1/N 2 I I [p(ij) - m] 2 (5) 

4 4 

N 2 

number of pixels of a subimage 

In a second step the pixels of a bloc are quantized to two levels by the 
mean m as threshold such as 1 if the pixel p(i,j) is greater than m or 0 
if p(U) is less or equal than m. 

Then m and v are quantized, both with the binarized bloc represent the 
coding information. 

For decoding such a bloc, we must first evaluate q, the number of l's of 
the binarized bloc, and then obtaining p = N 2 - q the related number of 
O's. 
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Then the two quantizer levels are computed (which preserve the mean 
and variance of the original bloc): 



Lo = « -^[v q/p] (6) 
Li =m +V [V p/q] (7) 

The last step consists in replacing each 1 in the binarized bloc with LI 
and each 0 with LO. 

In an ordinary form this algorithm permits to obtain a 2 bits/pel bit- 
rate, if mean and variance are quantized to 8 bits each and no other 
method is used to compact the subimage codes. 

The BTC algorithm is first very simple for implementation purposes and 
secondly reproduce very well sharp edges and impulsional texture like 
it is the case for echographic images. 

° Vector Quantization [9] 

Vector quantization was first introduced in the early 80's by Linde, 
Buzo and Gray [8], who developed a new approach of VQ design for the 
domain of speech coding. 

This algorithm is commonly called the LBG-algorithm and is still today 
a reference in VQ design. 

The base of this approach lays on the rate-distorsion theory of Shannon, 
which points out that one can obtain better performance with VQ than 
with scalar quantization in the meaning of distorsion and compression 
ratio. 

The basic principles lay on the operation to obtain an optimal codebook, 
which permits to encode in an accurate way a sequence of spatial or 
frequency parameters, instead of coding coefficient after coefficient 
like it is in scalar quantification. 

The VQ-operation consists in first assigning to a k-dimensional vector 
x=(xo,xi,...,xk-i) ( x in the spatial or a transformed domain) a channel 
symbol g(x) € M (set of channel symbols), which distance is the nearest 
to the encoding vector x, and this by an encoder g. 

At the receiver side the decoder 6 assignes to a channel symbol v € M a 
value of a reproduction alphabet A. 

If M has m elements then R = (log2 m) represents the rate of the 
quantizer in bits per vector and r = R/k is the rate in bits per symbol. 
Distorsion can be represented by the average euclidean distance 
between the original vectors and the reproduced vectors. 
In image compression vector quantization is either applied in the 
spatial domain by coding blocs of pixels or in a transformed domain 
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like DCT by regrouping in different ways DCT-coefficients to arrays 
and then coding them by the nearest vectors of a codebook [9] [10]. 

° Subband Coding [11] 

The basic of subband coding is to split up the frequency band of the 
image signal and then to downsample and code each subband separately 
or related, using a coder and bit-rate allocation closely matched to to 
the statistics of the considered frequency band. 

For this reason the characteristics of this method are those of a multi- 
resolutional spatial-frequency decomposition. 

Normally a 16 band system [11] is used for this purpose where FIR- 
filters are applied to approximate the ideal subband filter. 
Therefore the QMF (quadrature mirror filter) is used which cancels out 
the usual present aliasing (frequency gap) effect. 

For obtaining these 16 subbands the initial image is first partitioned 
into 4 bands. 
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Partition in 4 subbands by QMF 



Each of these four subbands is then demodulated to base band by a 
subsampling operation. 

For obtaing a 16-band representation each resulting band of the first 
filtering operation is partitioned again in four bands by QMF. 
For decoding inverse operations are done. 
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In a 2_D approach 
follows: 



QMF (separable product) can be represented as 



Hij(wl, w2) = Hi(wl)Hj(w2) 1< ij < 2 (8) 
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coding and decoding 

The subbands can be coded either using DPCM [11] with bit allocation to 
approximately minimize the mean square error , or by encoding for 
example the 16 subbands with vector quantization (exploiting 
depcncies between subbands) [12]. 

There are two other families of image coder, closely related to this 
method, called the Laplacian Pyramid Coding [13]and the Wavelet 
Coding [14] . 



3. Classification of medical Naecs im optimal coding 

purpose 

Radiological images are produced from very different kinds of image 
sources, being based on different physical principles. 
This causes that each type of image differs in a visual way concerning 
their basic textural appearence. 

For different types of images and diagnoses related to this modality, 
the radiologist either looks for example for edge-like details (stenose 
in angiography) or for texture-based details (tumors ) or for very small 
ponctual details (micro-calcifications in a mammography) p and so 
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image compression can create distorsion which could dctcrior this 
information. 

Thus it is important to define the best compression algorithms with its 
related compression ratio for each image class [26]. 

This can be done by a ROC-study with radiological experts showing 
different kinds of compression methods for the different image classes. 
The classification of radiological images which will be described in the 
further lines is essentially based on tcxtural features [21] of different 
natures: 

0 Gray Tone Spatial Dependence Matrice or Cooccurrence 
Matrice (GTS DM) [16] 

The method of cooccurrence matrices describes the image in function of 
a spatial dependance of neighboring pixels. 

Four matrices are computed for 0°, 45° > 90°, and 135° describing the 
transition probability for different directions. The evaluation of these 
matrices is based csscntialy on a first order neighborhood (eight 
nearest neighbors). 

The texture-context information is expressed by the matrice of relative 
frequency P(ij) with which two neighboring elements separated by a 
distance d ( d=l in our case )occur in the image, one with gray level i 
and the other with gray level j, in dependence of the angular 
relationship between the two neighbors. 

From these cooccurrence matrices following features can easily be 
computed : 

Angular second moment feature (which is a measure of homogeneity): 

fl =11 [ P(ij)/R ] 2 < 9 > 
Contrast feature : 

B«S n 2 { £ (P(ij)/R) ) (10) 
Correlation feature (linear dependencies): 

f3 ={ I I [ijP(ij) / R] - fix |xyj / (varx vary ) (11) 
J J 

R = normalization value 

|ix, \iy , varx, vary means and standard deviations of the 
marginal distributions associated with P(i,j)/R 
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To obtain rotation invariant features one can compute the first and 
second statistical moment of each feature for the different directions. 



° Directional Histogram (DH) [18] 

This method is based on a local edge decomposition using a convolution 
operator called Sobel-mask [27] and computing the gradient of the image 
in x and in y-direction (Ax, Ay). 

For computing the magnitude of the gradient we use the eucleadian 
measure: 

II AGII = >T Ax2 + Ayl (12) 

To take only into account the real edge-like features a threshold t is 
applied on the magnitude of the gradient such that : 

II AG II >t (13) 
and with L directional classes, where the direction is given by : 

0 = arctan [ Ay / Ax] + n/2 0= 0...JC (14) 

the angular class is found by : 

1 = int [0 * L/k] (15) 

the whole normalized directional histogram is represented by : 

Hd(k) = N0(k) / E N0(1) k = 0...L-1 (16) 

or Hd(k) = N0(k) / R 

where R is the normalization constant 

In order to extract rotation invariant features from the parameters of 
the directional histogram one can compute for example the circular 
cross-correlation or the second moment of the values represented in the 
histogram. 

The features itselves describe rather well the presence or absence of 
directional line-like characteristics ( probability of directional 
features )in an medical image ( angiography, echography). 
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0 Gray Level Difference Statistics (GLDS) [17] [19] 

This class of local properties is based on absolute differences between 
pairs of gray levels, and again sensitive for different directions .for a 
given displacement 3 = (Ax,Ay) let 

f3(x,y) = I f(x,y) - f(x+Ax, y+Ay)l (17) 

and let p3 be the probability density of f3(x,y), so that if m gray 
levels are present in the image then we have a m-dimensional array, 
where the ith component gives the probability that f3(x,y) = i. 
Classical features which can be computed of these parameters are 
contrast, entropy, angular second moment or inverse difference 
moments for example. 

With these directional sensitive features(0°, 45°, 90°, 135°), rotation- 
invariant features can be computed by taking the two first moments 
over all directions. 

The choice was made on these methods because they are easily 
computable. But other methods like power spectral analysis, gray level 
run length statistics or autoregressive models [20] [21] permit also to 
obtain equivalent texture discrimination. 

With ten of the above mentioned features we formed our parametric 
vector, which is used for clustering and classification purposes. 

° Classification [24] 

For classification needs we used a discriminant analysis 
[22][23][24][25], giving very interseting results from the point of view of 
image class discrimination. 

The goal of this classification method [25] is to find a transformation 
from a P-dimensional (P = 10 in our case) space IRp to a smaller 
M-dimensional (M = 2 in our case) space that retains most information 
of the initial space, or the most discriminant directions, and to obtain a 
good discrimination between the different classes. 

For a given sample set ( 40 images in our application) one can rapidly 
obtain an accurate estimate of distribution functions of the different 
reference classes (image classes) to formulate afterwards very simple 
decisicion rules in a dimension reduced space (M = 2 in our 
application). 

For this transformation [26] we used the P parameters of the textural 
features and we made a reduction to 2 factors. 
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The results obtained with the discrimant analysis turned around 99% 
for the discrimination ratio with a good projected variance (more than 
10% => stability of the clusters) for the different image classes. 

Composition of the vector: 

GLDM first and second moment of ASM,CONT,CORRE 

DH second moment and max. of the histogram 

GLDS first and second moment of ENTROPY 



Set of images for the learning phase: 




Results of classifications 



452 



4, Segmantation of radiological images £nj; compression 

purposes 

By analysing radiological images one can subdivide the image in 
different contextural regions like : 

- anatomic region 

- background region 

- text region 

For this different kinds of regions (after segmentation) it is interesting 
to apply the best adapted encoding methods, and by this way obtaining 
a more efficient coding. 
Why can this approach be efficient? 

By a study which was made with an hospital in Luxemburg, the results 
show that for certain classes of images ( MRI, urographies , thoraxes,...) 
the background component can take about 30% of the whole image 
surface. 

That means that by an efficient segmentation method [27][28]like region 
growing or a split and merge method one can easily fulfill an optimal 
discrimination between the real image data and the background 
information, because there exists an a priori knowledge that 
background is a very low entropy region (specially when impulse noise 
is eliminated by a median filter for example) and by this way 
optimizing compression in coding . 

Example: An image 1024 x 1024 x 8 bits representing 1 Mbytes of data 
, for which by using a block-DCT algorithm for [16 x 16] arrays, one 
can increase the compression ratio from for example 1:10 to 1: 14 by 
using a one bit flag for distinguishing if the bloc contents background 
or image information (background arrays are encoded with one bit). 
Background and text data (echography) can be very coarsely coded, by 
simply applying the average gray level for the background region and 
coding text information after binarization with run-length coding for 
example. 

For anatomic information a system as it was mentioned in the preceding 
chapter could be used. 

An other interesting consideration could be that the radiologist makes 
with a windowing function a segmentation of the anatomic region, to 
point out a primary and a secondary region. 

The primary region could content the pathological information and so 
very little or no distorsion can be applied to this region by means of 
compression ( implies a low compresion ratio) the secondary region can 



453 



be used for topological reference and so a more important compression 
ratio can be used for this region. 

When discussing this point with radiologists it seems that the 
windowing operation represents a minimum work for them before 
storing the final diagnostic images. 

5. Conclusion 

In this paper we first gave a short review of actual used compression 
algorithms , and we show that specially medical images represent the 
worst case for image compression at all. 

Therefore it is important to use image context and image class 
dedicated compression to allow an optimal compression with acceptable 
distorsion for the large field of image types which are present in 
radiology. 

The approach which was shown in this paper gives an idea of a possible 
solution for this kind of dedicated compression. 

But a very important requirement will be to construct afterwards the 
whole system in function of existing compression standards. 
In the future a decisive requirement will be to rely on international or 
de facto industrial compression standards within efficient self- 
adaptive compression systems. 
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Summary 

We present a software suite which incorporates a minimal, highly flexible data description with 
a set of associated manipulation programs and user commands. These may be used in a clinical 
or research environment in which there is a need for the storage* retrieval and manipulation of 
data from many original modalities, such as MRI, CT or SPECT. User-derived formats are also 
easily incorporated, such as Finite-Element Meshes or Colormaps. We have also developed a 
series of Graphical Applications which operate in an Object-Oriented, or Data-Driven, fashion, 
in order to make the use of these applications simple for the novice user. The applications 
typically revolve around the display of some part of an acquisition, which in turn may have 
been processed with the various software functions prior to display. 

1. Introduction 

In modern radiology, the clinician and researcher alike face the annoying situation in which 
various mutually exclusive tasks are required to perform a common operation, the viewing of 
images from different modalities, such as MRI and SPECT. Growing interest in the registration 
of multiple-modality studies makes it imperative that a reasonable solution to this dilemma is 
established. The high cost of multiple hardware platforms is one area in which a multi- 
modality-display technology could be expected to impact immediately. In order to manipulate, 
such as display, data from disparate modalities, it is necessary to design a suitable file 
format, attached as a report to each dataset, which suffices to describe the data therein 
properly and succinctly. There are a number of researchers and corporate bodies which have 
developed formats applicable to medical imagery and associated data. We will refer to two of 
these, the IMAGE format and QSH software system proposed by Noz and Maguire [1] and the TIFF 
format developed by the Aldus and Microsoft Corporations [2]. In developing the TR3 format, we 
investigated and compared both of the above approaches. 

The file format and software suite described in this paper were developed on computers running 
the UNIX 1 operating system, a common sytem which is gaining favor in all branches of the 
computer industry and therefore offers a large degree of hardware independence. The programs 
will run under BSD 4.3 and System V. With the exception of operating-system dependent programs 
such as shell scripts, most of the software may be easily modified to run under other common 
operating systems such as VMSr. (Another group within the university is developing software 
using the TR3 file format, running on networked PCs under MS-DOS 3 .) 



'UNIX is a registered trademark of AT&T. 

?VMS is a registered tradmark of DEC. 

^MS-DOS is a registered trademark of Microsoft Corp. 
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The following section, split into further sub-sections, describes the two parts of the file 
format, known as the HEADER and the TAG-LIST respectively, and provides the syntax used in 
functions which read and write TR3 files. Section 3 describes a typical example of the use of 
Our software suite with data in the TR3 format. The fourth section introduces some graphics 
applications and explains the data-driven approach taken in designing these applications. 
Finally, some discussion of the future of this environment, which is still evolving, will 
hopefully tie up any loose ends in the previous sections. 

2. TR3 Header and Tag List 

The format we adopted is known as a tagged format. This means that the data is partly 
described through the use of packets of information, found at the end of the file in the case 
of TR3, which contain data useful for data interaction other than the basic reading, writing 
and processing tasks which are expected to be the most common. Examples are patient name, 
physical voxel sizes, etc. The list of tags is always growing, and many are expected to be 
local to a particular site. 

The header is a short packet of information found at the start of the file, which also contains 
the image data and the tag-list. Rather than choose a scheme such as in the QSH [1], in which 
the header information is contained in a separate file from the data, and the header file also 
contains all the tags relating to the data, we decided that, as in the TIFF format [2], the 
header, data and tags should reside in the same file. The worry of the potential redundancy of 
the header as the result of loss of the data file, and vice versa, was the main reason for this 
choice. 

We now discuss the header and tags in turn, together with an explanation of the processes 
involved in opening a TR3 file for reading, writing and closing. 

2A. The Header 

It was decided that the header should contain only such information as was necessary for the 
essential operations such as reading, writing and processing the contents of the associated 
data. There are nine distinct values stored in the header, together with a tenth value which 
is used to check that the file is indeed in the TR3 format and is not corrupted. The first five 
values, or fields (as they are known in the syntax of the C programming language) are 



n_t, the number of entries in increasing time 

nz, the number of entries in increasing depth 

njr, the number of entries in increasing rows 

n x, the number of entries in increasing columns 

avals, the number of samples in each entry 



so that every dataset is assumed to have the four dimensions (or five if the last field is 
considered a dimension). In the case of a single image, the first two values would simply be 
unity. The data are assumed to be in row order. This simple description is surprisingly 
flexible; an example to which we shall refer later is a dataset in which there are ten gated 
frames of data, and each entry is in fact the concatenation of co-ordinates in 3-D space and 
three numbers representing ventricular perfusion, wall motion and wall thickness. This can be 
represented with the dimensions 

IT,Z,Y,X] = [Nriaroes,l,lJSlEnrries], [VALSJ = 6, 

IT^Y^X] = [Nfranx^VTCntries^, [VALS] = 1, 

depending on the preference of the application programmer. The data will be physically in the 
same order on disk storage in both cases. Conceptually, the former is perhaps more prefereble. 
As a check for validity, the next field in the header is 
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chksize, the product of the five dimensions. 

At the time a file is opened for reading, if 

chksize NOT.EQUAL.TO nfnz'n^nj^nvals, 

where the asterisk (*) repesents multiplication, then we can conclude that this file is not in 
fact in the TR3 format. This is a more robust check than a popular method in which a Magic 
Number, some known short sequence of characters of numbers, is placed at the beginning of the 
file and serves to (hopefully!) uniquely identify the format. Unfortunately, the check we use 
prevents us from exporting the UNIX command fifc(l), which attempts to find Magic Numbers in 
supplied arguments to the command, and, if found, print a message to the screen, but the 
multiplication is less prone to circumstance. 

The two fields that follow are used to interpret the data in order to define those functions 
which may meaningfully manipulate the data. The fields are 

fikjrype, the "interpretation" of the data and 

data type, the "interpretation" of each sample in each entry. 

Each is a concatenation of several sub-fields which identify particular details. In the case of 
the filctype, the sub-fields identify whether the data should indeed be interpreted as gated 
or simply a collection of related images, whether the data is grey-scale or in fact a 
pseudocolor representation (with an accompanying colormap hopefully), or even undisplayable 
data. One of the sub-fields is used to identify a modality or "type" of the data, which again 
may be locally defined as needed. In the case of the data type, the sub-fields identify whether 
the data is signed or unsigned, i.e. whether any of the data are less than zero, and whether 
the data is in integer or floating-point format, or whether the data is a "string" of 
characters with terminating symbols or just ASCII data. Simple arithmetic operations may be 
carried out to select or "mask" those parts of the fields which are not of interest, thus 
guiding the various functions which are called to process the data in any application. As an 
example, the function which scales displayable image data to occupy the full dynamic range of 
the input domain, must decide what the input domain is, and from that which scaling factor to 
use, in order to meaningfully scale the data. 

The final two fields in the file header are 

image_addr, the address of the first data point 

optionsjaddr, the address of the first tag, 

which are used to permit arbitrary data layout. The data is expected to be contiguous, but can 
be placed starting anywhere in the file. The same scheme applies to the list of tags. These 
fields permit various functions to quickly find any data or tag. 

2B. The Tag List 

In practice, it is very unwise to have a file layout other than 

HEADER followed by DATA foltowed by TAGS 

using the TR3 format. The reason for this is that the data itself is unlikely to change. In our 
system, in fact, processed data is always expected to be written to a new file. The tag-list 
for image files, on the other hand, is liable to grow as information is attached to the data. 
If the tags are not at the end of the file, new space must be generated which may involve 
moving the data "out of the way" to make room for new tags. This is a time-consuming operation 
for any operating system, so is to be avoided at all costs. The overhead, however, of simply 
reading in the tags at the end of the file as it is opened for reading, and then writing any 
new tag-list back in its entirety at file close time, is negligible. In this way, the tags are 
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saved in a format known as a "linked list" by the application program while the image file is 
being accessed, and if any new tags are added to the list, or if any tags are modified, the 
"new" tag-list in its entirity is written to end of the file, starting from the options_addr 
location. The job of finding the extra space at the end of the file is left to the operating 
system - in general, this process involves a negligible delay. The tags themselves contain 
three fields, which are 

tagjabel, the "name" of the tag, actually an integer constant 
datatype, the "type" of the tag data (as for header above) 
nekments, the number of items of the specified type. 

This format is similar to the TIFF tag format The major difference between TR3 tags and TIFF 
tags, however, is that in TIFF, the philosophy is to "simply follow the pointers, wherever they 
may lead" [2] because every tag has a field in it which identifies the starting address of the 
data for that tag, which may be anywhere in the file. This scheme of course makes expansion of 
the tag-list straightforward, but makes the file layout complex. As well as this, all file 
information, including dimension and type definitions, are represented as tags, which makes 
file processing more involved. A useful advantage of having the file "laid out" in the TR3 
format is that it is easy to turn the data back into a "raw" state if necessary (for instance, 
if the data is to be converted to some unusual format, or if the header information or tags are 
incorrect). 

The overhead of reading in the entire tag-list (and possibly writing it all out again) may seem 
considerable; in practice, the size of the tag-list is unlikely to exceed a few percent of the 
data, so the cost of keeping this information online is in fact low. 

2C. Reading and Writing TR3 files 

The TR3 software is known in UNIX parlance as a "library", and using these routines in a 
program is known as making "library calls". The library calls used to open and close TR3 format 
files are arranged in a similar way to the standard UNIX library calls fopen and fclose. They 
are in fact functionally almost identical to the calls found in the QSH library, and have the 
same names. These names were in fact originally part of the University of North Carolina's VSH 
[3] library. The values returned by the calls reflect the success or failure of the call. For 
instance, the following lines indicate a call to open a TR3 file in order to read its 
contents: 

#mdude <imagchead.h> 

Improg Header *imphd = tmopeii("testJmg > > READ); 

If this call succeeded, the variable imphd would now have all the necessary information about 
the TR3 file TESTJMG. If the call failed, however, the returned value would the NULL, or zero, 
value, which is returned by the UNIX library call fbpen(3s) upon error. In addition, the 
following TR3 call could be used to assess the reason for any open failure: 

#indude <imageheadJi> 
int status = tmdiag(0); 

The integer status would have a value which would reflect a particular error encountered during 
the last TR3 file operation, such as a value indicating that the file testimg did not exist. 

Opening a TR3 file for writing is a more complex operation. It is of paramount importance that 
the given filename does not exist, otherwise if the user had write permission to the file, it 
could be truncated to zero length by UNIX at opening time. So the call tmcreatO must 
necessarily behave slightly differently to the UNIX system call creat(2). The function checks 
that the named file does not exist before creating the new file. The following call attempts to 
open a particular file for writing: 
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#indude <bnagebeadJi> 

short filetype = SIMPLE DATA | SINGLE IMAGE | GREY LEVEL; 
short datatype = CHAR J UNSIGNED; 
kmg dimensions] = {1,1,256,256,1}; 

Improg Header •imphd = imof»t(*tcstiiing" T DEFAULT, filctype^tatype4imcoskios); 

where the words in uppercase indicate constants* The DEFAULT constant allows the owner of the 
file and other specified users known as the user's group to read and change the file (i.e. it's 
tag-list) as required. If this call succeeded, the file would be set to contain a single set of 
data representing a 256 by 256 pixel image, where each entry is one byte in size, and is 
interpreted as unsigned, in the range [0-255]. Again, if this call failed and returned a NULL 
value, a call to imdiag( ) would supply the cause of the failure. 

3. Object-Oriented Images 

Any display system which makes the display and analysis of images an arduous task for the 
clinician is unlikely to be popular. Many hospitals purchase computer systems which 
subsequently are rarely used because of the sheer effort required to use them effectively. It 
is important that in any user-oriented application, the choices available are understandable 
and are the only choices which can be made given the present data. For instance, the option to 
display the 3-D-cooidinates/ventricuIar-paramaters example discussed in the previous section is 
meaningless and would be confusing if it were offered. The applications developed for use with 
TR3-format data have been expressly designed to allow only options which are meaningful at that 
moment, to reduce confusion as far as possible. The applications included allow the user to 

A) extract a 4-D subvolume from an acquisition 

B) interactively segment parts of interest in a 4-D set 

C) cine gated sequences 

D) make physical measurements on displayable sections 

E) overlay MRI images with SPECT for registration 

F) construct and display wireframes from MRI and SPECT. 

In each of these applications, the emphasis has been that the user can only apply those 
functions which are meaningful at that time. In the wireframe rendering application, the user 
could select as input any file with 3-D-coordinate/ventricular-parametors data, but would not 
be able to "find'' i.e. see, choices which contained e.g. MRI data. 

The particular environment we have worked under is the SunView* window system. With the rapid 
rise in popularity of the window system, under which we are also developing software, a 
largely hardware-independent windowing environment, is available in which to run simple-to-use 
medical imaging applications supporting multi-modality data. 

4. Conclusions 

We have presented a brief description of the TR3 image file format and software environment. 
The emphasis in the development of TR3 has been 

1. Simple, extensible image descriptions 

2. Data-driven user applications. 



^SunView is a tradmark of Sun Microsystems Inc. 

5 X is a trademark of Massachusetts Institute of Technology. 
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It is hoped that these goals have been properly explained in this paper. We believe they have 
been met in general with TR3. The library, and some of the key concepts, are still fluid 
however, as certain factors become of temporary importance to users and programmers. The 
commands available to the user grow but it must be borne in mind that an overly-rich 
environment will tend to cause confusion, so the list is limited as far as possible. The 
software has been tested on computers from Sun, Stardent and IBM. There is support for 
conversion to and from a number of other formats, including TIFF. Further descriptions of 
application software will be presented at a future date. 
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Abstract 

This paper compares a particular surface rendering method with a specific volume 
rendering technique. The former has not been published in the literature and hence is 
described in some detail. The latter constitutes a published work and the comparison is 
based on our own implementation of this work as close to its original form as possible. 
The bases of comparison are: ability to portray thin bones, clarity of portrayal of sutures, 
fractures, fine textures, and gyrations, smoothness of natural ridges and silhouettes, and 
time and storage requirements. Our preliminary conclusion is that the surface method has a 
slight edge over the volume method for the portrayal of information of the type described 
above and a significant advantage as far as time and storage requirements are concerned, for 
implementations in identical environments. 

Introduction 

Currently there are two classes of approaches, known as surface rendering and volume 
rendering, available for visualizing and analyzing medical three-dimensional images. In 
surface rendering, the premise is that the given image contains data pertaining to certain 
tangible structures in the human body, and hence, that each structure can be visualized 
by its surface estimated from the image. In volume rendering, there is no assumption of an 
underlying structure or its surface, rather the tissue material of each structure to be visualized 
is estimated in every voxel and the structures are visualized through radiocity calculations 
based on the opacities and colors assigned to different tissues and on some assumed optical 
behavior of tissues. In both methodologies, the important issue is how to do rendering so 
that medically relevant information is portrayed in the rendered image. 

Recently there has been a spurt of activity related to volume rendering and its medical 
applications. Many claims of superiority of volume over surface rendering have also been 
made the validity of which is neither obvious to us nor has been established scientifically. 
Unfortunately, some claims have been accepted as established facts and have started appear- 
ing in the standard literature. The comparative assessment of surface and volume methods is 
a very complex subject It is not possible to derive scientific facts on this subject indepen- 
dent of the organ system under study, the imaging modality and its parameters, the actual 
application, the algorithms used in each methodology and their parameters, and the observer. 
The scope of our study is, by necessity, limited, but hopefully it will initiate further studies 
of this nature on this important subject 
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For surface rendering, the methodology used is our own [1], and for volume rendering, 
we use the technique reported in [2]. Since the main objective of this paper is comparing 
the methods, we will not delve into their description. Our comparison is based on our 
implementation of the methods in a common computing invironment. 

Comparison 

We use two data sets for comparing the renditions created by the two methods. The 
first is a 512x512x62 scene pertaining to a trauma victim obtained via CT with a uniform 
pixel size of 0.4 mm and a slice spacing of 1.5 mm. The structure of interest here is the 
patient's skull. The second data set is a 256 x256 x32 scene of a human brain specimen 
obtained via MRI. The pixel size and slice spacing for this data are 0.78 mm and 3 mm. In 
creating renditions, we have used identical values for the two methods for the parameters 
that are common to the two methods. All images are created without any antialiasing. We 
have used three different "resolutions" for the rendered images. For medium resolution* we 
assume that the voxel size aj for the interpolated scene is equal to the pixel size an in 
the input scene. For high resolution, we assume that a/ = | an and for low resolution 
a i = 2a r. For all resolutions and for both methods, the size of the two-dimensional array 
defining the image is determined such that the projection of a single boundary face for any 
orientation of the object of interest falls just inside a single pixel of the image. 

In an attempt to determining what should be the optimal values for the parameters of the 
classification function used in volume rendering, we have created literally hundreds of images 
corresponding to some chosen view of the structure by varying the values of the parameters 
continuously over an interval whose end points in our view represented extreme situations. 
We do not claim that our choice of values for these parameters is the best possible, yet this 
arduousness signals a difficulty which indeed is a counterpart of the difficulty of segmentation 
in surface rendering. 

Figures 1-4 show some renditions derived from the two scene data sets described earlier. 
Figures 1 and 2 show identical views generated from the first data set by (a) the surface and 
(b) volume methods at medium (Figure 1) and low (Figure 2) resolution. Figure 3 shows 
identical views generated from the second data set by the two methods at high resolution. Our 
following comparative statements are based on these and other images we have generated and 
on our experience with the two specific implementations of these methods. We emphasize 
that there was no interactive editing of any sort involved in the creation of any of these 
images. 

Thin Bones: Depiction of thin bones, especially those in the vicinity of the orbits, is 
equally poor in the two methods. There are two main reasons for this difficulty. First, the 
characteristics of the scene in the vicinity of thin bones are very similar to those of the 
skin. Therefore, except when dry skull specimens are used, it is just about impossible to 
segment for surface rendering or classify for volume rendering voxels constituting thin bone 
differently from those constituting skin using global criteria such as those used in this paper. 
More sophisticated neighborhood operators have been studied [3] for segmenting such voxels 
with some success in scenes of lower resolution than that used here. In our implementation of 
this technique, the improvement achieved was negligible, perhaps because the neighborhood 
characteristics responsible for the earlier success are no longer valid in the higher resolution 
scene. The second reason is that even when thin bones can be segmented/classified effectively 
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(say using some specialized knowledge), their effective rendering is difficult because the 
normal N computed from an estimate of the gradient in the vicinity of thin bones is not 
reliable. In fact, when we modified the surface rendering algorithm to incorporate some 
specialized knowledge such as specifying the approximate region in which thin bones may 
lie and to use features in addition to density, we were constantly mislead by the rendering 
because of unreliable N. Only when we resorted to distance-only shading, we saw the 
improvement in the detection of thin bones resulting from the modified algorithm. These 
problems constitute important research topics in both surface and volume methods. 

Sutures, Fractures, Fine Textures: There is no clear advantage in either of the methods for 
the depiction of these features. In our view, the surface method produces sharper depiction 
of the features than the volume method. 

There is a certain amount of tolerance and robustness in the surface method used here 
which are responsible for the effectiveness of the depiction of fine features irrespective of the 
digital nature of the surface. For example, for a suture of a skull to be portrayed effectively, 
it is not necessary that the feature be modelled precisely in the digital geometry of the 
surface. All that is required is that the feature be in the close vicinity of the modelled 
surface. Appropriate interpolation and normal estimation procedures guided by the geometry 
of the surface can bring out the feature in the rendition. In fact, the rendition using even a 
thresholding method is surprisingly insensitive to the actual threshold used. Figure 4 shows 
a surface detected at a threshold value 60 Hounsfield Units above that used for the surface 
shown in Figure 1(a). 

Gyrations, Ridges, Silhouettes: The silhouettes and natural edges appear to be portrayed 
smoother by the surface method than the volume technique. Though the underlying surface 
geometry is digital and hence jagged, because of the method of estimating N and of 
determining the real (rather than digital) location of boundary faces [1], the display of ridges 
and edges does not show jaggedness even without the antialiasing operation. The gyrations 
of the brain surface are portrayed equally well by the two methods. In general, the surface 
method seems to show the 'slicing artifact* more prominently than the volume method, 
perhaps mainly because, in the latter, such artifacts are smoothed out by the integration 
process. This phenomenon, in our view, has an undersirable effect namely that subtle natural 
textures in the surface are portrayed poorly in the latter method (See Figure 1). 

Tune and Storage: It is not possible to give a precise comparison of speed of rendering 
for the two methods since our implementation of the volume method is much less optimized 
compared to the surface method. Two orders of magnitude speed up over a straightforward 
implementation has been achieved for the volume method [4] using a variety of techniques 
whose objective is to minimize the number of rays cast and the number of voxels sampled 
along each ray without sacrificing image quality. Based on the reported timings, our estimate 
is that, for the current state of optimization for the two methods and implementation in an 
identical environment, the surface technique is 25-30 times faster than the volume method. 
The number of surface elements (1.2 million for the example of Figure 1(a)) is usually much 
smaller than the total number of voxels that enter into computation in volume rendering. 
Further, the actual computations done per element during rendering are fewer and simpler in 
the former method than in the latter. In our implementation, the preprocessing time required 
by the former method is roughly 1/3 that required by the latter. 
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The storage requirement during the rendering process is determined mainly by how 
the algorithms are implemented. In general, as the implementation is made more and more 
optimal for speed, the memory requirement increases and vice versa. The volume method 
requires enough space to store two scenes in addition to the space required to store the image 
being generated. The surface method requires random access only to an image and a z -buffer 
and not to the surface data, and therefore the memory required is determined by the size of 
these buffers. With these considerations, the CPU memory required during rendering by the 
surface method for the example in Figure 1 is about 20 times less than that required by the 
volume method, though this factor for disk space requirement is about 3-4. 

Finally, in our experience, the volume method is easier for a straightforward first 
implementation than the surface method 

Conclusions 

From purely a technical viewpoint, we consider that the surface rendering method of 
the type described in [1] should be preferable to the volume method considered in this paper 
[2] on the basis of the nature of the information portrayed in the renditions and time and 
storage considerations. As we tried to improve our implementation of the two methods it 
became clear to us that the differences between them start disappearing . Therefore it is just 
as relevant to compare between these methods as it is to compare techniques in the same 
methodology. In our view, the sources of the shortcomings in the two methodologies are 
common. There is a great deal to be benefited by each methodology from developments 
in the other since a new development in one has often an appropriate parallel connotation 
in the other. For example, it is possible to combine some of the ideas related to normal 
estimation and interpolation used in surface rendering with the volume rendering method. 
We could have done even a connectivity analysis to discard the clutter appearing in the volume 
rendition examples. Then the differences between the two methodologies start disappearing 
and only their common weaknesses remain. The bottom line, in our view, that decides 
which methodology is superior in a particular visualization task is which is more effective, 
segmentation or classification, in successively identifying tissue regions and interfaces. If 
situations exist where there is such a difference, the design of appropriate techniques that 
yield the difference is an interesting research topic. 
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Figure 1. Renditions derived from the first scene at medium resolution: (a) 
surface rendering, (b) volume rendering. 
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Figure 2. Renditions as in Figure 1 but computed at low resolution and different views. 
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Figure 4. Surface rendition as in Figure 1(a) except for 60 HU higher threshold. 
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Due to the effectiveness of mammographic screening in 
the reduction of mortality from breast cancer 1 * 2 , we are 
investigating various radiographic features of the typical 
mammographic examination. Our principal efforts have 
been towards the development of schemes that aid 
radiologists in the detection of breast cancer. This paper 
reviews two radiographic features of masses that radiologists 
use in the detection of breast cancer: the expected architec- 
tural symmetry of the left and right breast images of the 
same view (e.g., cranio-caudal or medio-latcral-oblique) 
and the presence of spiculation along the mass margin. 

The analysis of architectural symmetry between the left 
and right breasts has been adapted by radiologists as a useful 
tool in detecting masses in mammograms^. Radiologists are 
taught to visually compare corresponding parts of the left 
and right mammograms in order to inspect isolated selected 
regions for possible abnormalities 3 . Our computerized 
detection scheme is motivated by this systematic method of 
viewing mammograms. 

In current clinical practice, after a mass is found by a 
radiologist, a subjective likelihood of malignancy is 
determined by visual inspection of the mammogram. One 
of the primary radiologic features used in differentiating 
malignant and benign masses is the amount of spiculation 
of the mass, Our computerized scheme attempts to quantify 
the degree of spiculation of detected masses in order to aid 
in their classification. 




Introduction 
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Methods 

We arc developing a nonlinear bilateral subtraction 
technique for the computerized detection of masses in 
screening mammograms. In this scheme, gray-level 
thresholding is performed on corresponding digital images 
of the left and right breasts prior to digital subtraction 
using a "left-minus-right" convention. After subtracted 
pairs of various threshold images are obtained, the 
resulting data are combined into two images that contain 
information regarding suspicious areas in both breasts. 
Feature-extraction techniques are then applied to reduce 
the number of false-positive detections. 

Our computerized scheme for the classification of detected 
masses measures the amount of spiculation exhibited by the 
mass in question. First, the mass is extracted from the 
anatomic background of the mammogram. In this study, the 
extraction procedure was performed either by having a 
radiologist manually outline the mass or by having the 
computer track the border from a binary image obtained 
using gray-level region growing. Spiculation can be 
considered a fluctuation about the border of a mass. After a 
mass is isolated from the background, its margin is smoothed 
in order to substantially reduce possible spicules. The 
difference before and after the smoothing operation 
provides an indicator of the magnitude of fluctuations, with 
a large difference corresponding to a high degree of 
spiculation and therefore, a greatly increased likelihood of 
malignancy. Two measures of spiculation are the standard 
deviation of the fluctuations about the mass border, and the 
normalized difference in the areas of the mass with and 
without the presence of spicules. 

Results 

The detection scheme was examined using 46 pairs of 
clinical mammograms (18 patients with a mass and 5 patients 
without a mass). The screen/film mammograms were 
digitized with a TV camera/Gould digitizer yielding a 512 by 
512 by 8-bit image matrix with an effective pixel size of 0.4 
mm. The presence of masses in the images had been verified 
by surgical biopsy. This preliminary study yielded a true- 
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positive detection rate of approximately 95% with 3 false- 
positive detections per image on average. 

The scheme for quantitation of spiculation was examined 
using 35 mammographic masses. The pathology (malignant 
or benign) of each mass had been determined from surgical 
biopsy. Screen/film mammograms were digitized using an 
optical drum scanner with a 0.1 mm pixel size and 10-bit 
quantization. The spiculation measures obtained using hand- 
drawn outlines correlated well with those obtained from the 
computer-determined outlines. Also, the calculated measures 
of spiculation agreed well with a radiologist's subjective 
impression of the degree of spiculation exhibited by the 
masses. 

Our preliminary results suggest that computerized 
detection and classification schemes may aid radiologists as 
a diagnostic tool. 
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I, INTRODUCTION 

Quantitative analysis of angiograms, two-dimensional images of the 
vascular system formed by the projection of x rays through blood vessels into 
which an iodine contrast agent is being injected, may provide useful morphologic 
and functional information on the vascular system for the diagnosis of vascular 
diseases. Computerized methods have been developed for quantitative measurement 
of vessel sizes in angiograms. 1-6 However, neither absolute vessel size nor percent 
stenosis directly reveals the functional significance of stenosis, i.e., its effect on 
blood flow. Thus, measurement of flow rates would be useful for evaluation of the 
significance of stenosis. In addition, flow rales could be employed to determine 
coronary flow reserve and to evaluate the effect of interventional procedures. 

Several investigators 7-11 have proposed methods for the determination of 
blood flow rates from angiograms. The flow rates were calculated from the vessel 
cross-sectional area and the velocity of the bolus which was estimated using the 
time required for the bolus to traverse the distance between two locations in the 
vessel, i.e., the time of transit. The time of transit was estimated as the difference 
between time parameters calculated from time-density curves which were obtained al 
specified locations in the vessel. Time-density curves represent the concentration of 
contrast material at a given location in the vessel as a function of time. For steady- 
flow conditions, average flow rates have been calculated to within 5% of the true 
flow rates. 8 * 11 However, for pulsatile-flow conditions, such as those in arterial 
flow, the average flow rates calculated from time-density curves have been found to 
deviate from the true average flow rates by as much as 100%, and the average error 
in the calculated flow rates is approximately 20%. 7 ^' 11 

In this study, we present a new method for determination of blood flow 
rates from angiograms which is based on analysis of the temporal dependence of the 
spatial shift of the contrast material in opacified vessels in angiograms acquired as 
the bolus of contrast material proceeds through the vessel. In our method, time- 
density curves are not employed, rather the flow rates are calculated from distances 
determined by comparison of curves (which will be called distance-density curves ) 
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which represent the concentration of contrast material as a function of distance in 
the vessel. 



II. MATERIALS AND METHODS 

Digital images of a vessel phantom were obtained using a Digitron 2 DS A 
system with a matrix size of 512x512 and a pixel size of 0.36 mm. 11 The image 
data from the Digitron 2 were analyzed with a VAX 1 1/750 computer connected to a 
Gould-DeAnza FD5000 image processor and CRT display. The vessel phantom 
consisted of non-distensible tygon tubing with a 6.7 mm nominal inner diameter 
oriented parallel to the imaging plane of the image intensifier. The phantom was 
imaged with a 9 cm thick lucite plate in order to generate scatter similar to that of 
the body. A pulsatile pump was used for supply of a 0.9% normal saline solution 
through the phantom. The pulsatile frequency of the pump was maintained at 
approximately 1 Hz, and experiments were performed for average flow rates ranging 
from 3.5 to 10 cc/sec. Instantaneous as well as average flow rates were measured 
with a 6.7 mm extracorporeal flow probe and a Cliniflow electromagnetic flowmeter 
equipped with a strip chart recorder. Images were acquired at 15 frames/sec as an 
injection of 2 cc of Renographin was delivered with an Angiomat 3000 injector over 
0.25 to 0.5 sec at a distance of approximately one meter from the imaged section of 
the vessel. 

After digital subtraction, a vessel region was manually selected for 
analysis, and the centerline of the vessel was determined. The vessel size and 
contrast were determined along the length of the vessel. The cross-sectional area of 
the vessel was estimated from the vessel size by assuming a circular cross section 
for the vessel. The vessel contrast was determined as the difference between the 
average pixel value in the central region of the vessel and the average pixel value in 
the adjacent background region. From the vessel size and contrast, the distance- 
density curve was calculated for each angiogram. We found that the shape of the 
distance-density curves remains fairly constant over short time intervals as the bolus 
proceeds along the vessel. Thus, the distance that the bolus moves along the vessel 
between two image acquisitions can be determined by comparing the respective 
distance-density curves. It should be noted that the shapes of the time-density 
curves have been observed to change as a function of distance along the vessel. 7 * 12 
The distance-density curves were compared by shifting one of the curves spatially in 
an iterative manner until the root-mean-square difference between the curves was 
minimized. The spatial shift which yielded the minimum root-mean-square 
difference was considered to be the distance that the bolus traversed between the two 
acquisitions. The "average" flow rates between the acquisitions were then calculated 
from this distance, the frame rate, and the cross-sectional area of the vessel. Thus, 
for high frame rate acquisitions, "instantaneous" blood flow rates can be determined. 
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III. RESULTS 

Figure 1 shows the instantaneous flow rates (circles) calculated using the 
distance-density technique from a study in which the average flow rate was 5.0 
cc/sec. For comparison, the instantaneous flow rates measured by an 
electromagnetic flow meter arc plotted here as the solid line. For the fourth pulse 
and the first portion of the fifth pulse, the "instantaneous" flow rates calculated from 
the distance-density curves appear to agree well with those measured by the 
electromagnetic flow meter. For the second pulse, the apparent disagreement 
between the flow rates obtained with the two techniques may be due to laminar flow 
effects and saturation effects. In the case of laminar flow, the flow rate in the center 
of a vessel is twice the average flow rate across the vessel lumen. It should be 
noted that, for the first portion of the second pulse, the calculated peak flow rate is 
approximately twice the peak rate measured with the electromagnetic flow meter. 
Thus, these results indicate that the calculated flow rates may correspond to those in 
the central region of the vessel lumen. The flow rates during the time between 1.0 
to 2.2 seconds were probably underestimated because the vessel was at maximum 
opacification for this phase, and thus, the corresponding distance-density curves were 
nearly saturated. 

For the studies performed, we have determined that the instantaneous flow 
rates measured with our technique agreed with those measured with an 
electromagnetic method to within an average of 2.3 cc/sec for pulsatile flow 
conditions with peak flow rates of up to 20 cc/sec. In addition, the average flow 
rates were calculated from the slope of the straight line which best fit the curve 
representing the cumulative distance traversed by the bolus as a function of time. 
The average flow rates determined using the distance-density and electromagnetic 
methods agree to within an average of 1 1% with a maximum deviation of 28%. 
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Figure. 1. Comparison of the instantaneous flow rates calculated from the distance- 
density curves ( • ) and those measured with an electromagnetic flow meter (solid 
line). The average flow rate was 5.0 cc/sec. 
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Summary: 

In a series of 16 cases 3D-models after matching their overall dimensions and 
structural details to actual anatomical measures were used for surgical planning 
and simulation as well as predominantly for fabrication of large individual allo- 
plastic implants for cranioplasty. The model mediated implant manufacturing 
finds increasingly more indications with far better overall results in aesthetics 
and quality of cranioplasty, since models in true dimensions can be made avai- 
lable in short reconstruction and milling intervals of about one week. 

Introduction: 

Preliminary experience on 3D-based computer assisted manufacturing of in- 
dividual alloplastic implants for cranial and maxillofacial osteoplasties has 
already been presented on the CAR 4 89 meeting in Berlin [7]. 3D-imaging is be- 
coming increasingly successful in clinical applications. VANN1ER et al. [8] de- 
monstrated that premature synostoses were more easily diagnosed by use of 3D- 
imaging, while normal x-ray was equally effective in detection of pelvic fractu- 
res. Knee injury diagnostics have been improved by three dimensional display 
of mathematically disarticulated objects [4]. In orthopedics CAPS - computer 
assisted prostheses selection - is being applied to facilitate the right choice of hip 
endoprostheses selection [1] and in surgery of cleft palate and cranial deform- 
ities MARSH and VANNIER already some years ago pointed out the usefiillness 
of 3D-techniques for surgical planning [6], which was recently improved by so- 
phisticated 3D-disarticulation methods [11]. TOENNIES et al. [9] demonstrated 
the use of surface registration for the segmentation of implanted bone grafts. 
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ULLRICH and MATHEWS assessed the use of 3D-CT in 100 cases of facial 
fractures [10] and Craniofacial applications were evaluated recently by Mc- 
EWANetal. [2]. 

However, the rendering of 3D-images for surgical planning and diagnosis in 
most image processing systems like CEMAX, PIXAR, KONTRON etc. does not 
display 1:1 real size 3D-images, thus making calculation of conversion multipli- 
cators necessary. Despite fabrication of 3D-alloplastic individual implants pose 
little technological problems, at least since the feasibility was proven years ago, 
there was no decisive increase in application of this technique. The professional 
application has been restricted to some singular cases due to problems of mat- 
ching visualization and 3D-milling and to guarantee real size implants. Most of 
all due to a lot of detail problems which are often more or less organizational and 
coordinational difficulties. In principle two methods are known: 

• direct implant milling right after designing in a CAD system, 

• indirect, model mediated 3D-implant generation. 

In the mean time the number of clinical applications and indications has in- 
creased. 

Methods: 

CT data were generated on CAT scanners SIEMENS SOMATOM-II or -DRH 
with 256x256 or 512x512 matrices. Five second slices were recorded at 1 or 2 
mm intervals. Data were stored in compressed mode, single density, on 8 inch 
floppy disks. The disks were sent to HOWMEDICA GmbH, Kiel, Germany, for 
reconstruction and milling of models. 

To reconstruct bony defects the CT image data were segmented by threshold 
methods based on Hounsfield units. For thresholding a suitable interval was se- 
lected. The image matrices were transformed in binary matrices according to this 
Hounsfield interval representing the bony structures of interest. 

The contour of the bone structures were detected by a recursive algorithm. 
The algorithm first traces the boundary, then the inside of the boundary to detect 
holes using the dualism of inner and outer contours. In particular the algorithm 
builds a data-structure (contour tree), which reflects the appearance of holes and 
branches. 

The geometrical model of the bone structure was generated by matching the 
contour tree elements obtained from adjacent slices. This surface of the bone 
structures was reconstructed out of two matched contours under the constraint, 
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that the surface of the model was minimized (CHRISTIANSEN et al. [3], 
FUCHSetaL [5]). 

Using the MEK-system (MEK: Medizin-Elektronik GmbH, Kiel, Germany), 
a life sized 1:1 polystyrene or polyurethane model was prepared by a 5 axle 
milling machine. 

Results: 

In the original setup a segmentation of bony contours was done with a Houns- 
field value of 190. These parameters, however, did not represent patients true 
bone stock, rather overestimating skull thickness by 20-30%. By comparison of 
fresh anatomical skull specimens dimensions with those of the milled models, a 
standardization of the method was achieved to give a close matching of overall 
model sizes and detail skull thickness to actual patients dimensions. Finally two 
different Hounsfield parameters were used to describe the outer and the inner 
skull cortex borders. 

A. : Planning of surgical procedures was done in polyurethane models in: 

• mandibular hypoplasia, 

• facial skoliosis, 

• posttraumatic facial asymmetry, 

• Apert syndrome, 

• hypertelorism and craniofacial stenosis, 

• carcinoma of floor of mouth infiltrating skull base in middle cranial fossa. 
Planning of surgery with help of models was rated very successful - the model 

providing considerable help to the surgeon in preshaping of contours, determi- 
nation of osteotomy sites allowed for improved surgical results. 

B. : Preoperative simulation of tumor surgery was done in: 

• meningioma en plaque of the left sphenoid wing, (C. R.) 

• meningioma of the right sphenoid. (D.R.) 
Preoperatively the tumor masses were excised in the models; the respective 

defects were pre-Op closed by wax modulations according to the unaltered con- 
tralateral structures. Wax modellations were converted into polymethylmetha- 
crylate (PMMA) bone cement implants or into the new IONOS glass-ionomer 
bone cement implants using impression techniques. 

Pre-Op TU excision was simple in the case of very clear tumor borders in 

C. R. (Fig.l). Though the defects contours created in the model did not match the 
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definitive osteotomy size, an intraoperative shaping of the prefabricated glassio- 
nomer implant was easy. The implant was glued into the left temporal defect thus 
reconstructing subtemporal and temporal bone and lateral orbital wall (Fig.2). In 
the D.R. case with more diffuse TU-infiltration of sphenoid wing, orbital roof 
up to the optic canal, and the sphenoid sinus delineation of TU-borders and pre- 
operative definition of extent of bone excision proved more difficult A glassio- 
nomer implant replacing subtemporal bone and part of lateral orbit was prefa- 
bricated. Intraoperative reshaping of the implant to a small extent and adjustment 
to the craniotomy defect were necessary, the skull-implant was then glued into 
the defect reducing general operation time considerably. 

C: Alloplastic cranioplasties were performed in the following cases: 



left frontoparietal, 


(L.V.) 


bifrontal, medial, 


(L.I.) 


bifrontal, medial, 


(M. H.) 


right fronto-orbito-parietal, 


(E.C.) 


right parietal, over midline, 


(W.R.) 


bifrontal, medial, 


(B.H.) 


huge, bifronto-biparietal. 


(L.T.) 



Implants were molded and sculptured in wax according to symmetry and with 
regard to preoperative patients appearance. Using plaster of Paris molds and im- 
pressions were made and wax specimens converted into implantable bonecement 
skulls using PMMA or IONOS R glassionomer cement, either. PMMA-specimen 
were sterilized by immersion in polyiodine solutions for 48 hours, IONOS R glas- 
sionomer-cement specimen were sterilized by autoclaving. 

Skull reconstructions were extremely improved using 3D-prefabricated im- 
plants of either IONOS R glassionomer cement or PMMA. Frontal-parietal and 
temporo-lateral aspects were nicely contoured to general skull symmetry. Espe- 
cially difficult fronto-orbito-parietal reconstruction, including supraorbital rim, 
part of the orbital roof and continuity to fronto-zygomatical process of zygoma 
were reconstructed to a high degree of standard and craftmanship. Fixation of 
alloplastic implants was accomplished by either suturing or using the adhesive 
properties of the new IONOS R glassionomer cement, which in tensile testing ex- 
hibited an interfacial tensile strenght of 1 - 2 N/mm . So far no complications, 
e.g. no infections, were encountered in the present series. 




Fig. 2. : Fitting of individual IONOS glassionomer 3D-implant in temporal cra- 
niotomy defect. The implant is glued into the defect 
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Conclusion: 

Though at a first glance the model mediated approach is not the most elabo- 
rate and advanced one, at least providing the suigeon with a 3D-polyurethane- 
model with matched dimensions to true patient measures, at the moment is a de- 
cisive step foreward in the state of the art of alloplastic cranioplasty and bone- 
tumor surgery of the cranium. Precision of plastic reconstruction is increased 
with aesthetic improvement in shaping of the cranium to natural symmetry. With 
regard to advances in microsurgical techniques an equivalent and better quality 
of cranioplasty is reached. Furthermore the technique of 3D-reconstruction fa- 
cilitates application of implantologically desirable materials. Once the method 
was installed with reliably short intervals of milling actual sized matched models, 
there was an increasing demand noticed. Efforts should be made to further faci- 
litate model fabrication or to enable direct milling of dimension-matched im- 
plants, eventually rendering model-making obsolete. 
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1 . Introduction 

Fluid mechanics is one of the fields that branches out into various specialities 
all the way from aerodynamics to medicine. It deals with the statics, kinematics, 
and dynamics of fluids, since the motion of a fluid is caused by unbalanced forces 
exerted upon it. In this paper, we will be concerned with some of the dynamic 
properties of liquids. The static properties of fluids will not be dealt with. We will 
describe a flowing liquid by a flow field. The mathematical description of the state 
of a moving fluid is effected by means of its velocity v = v(x,y,z;t), the pressure 
p(x,y,z;t) and the density p(x,y,z;t). If we are given the three components of the 
velocity v, the pressure p and the density p, the state of the moving liquid is 
completely determined. It should be remembered that all the qualities are, in gen- 
eral, functions of the coordinates x,y,z and of the time t. We emphasize that 
v(x,y,z;t) is the velocity of the liquid at a given point (x,y,z) in space and at a 
given time t, i.e., it refers to fixed points in space. The same remarks apply to p 
and p. In this paper, the spatially resolved characterization of a flow field will 
mean the measurement of v as a function of (x,y,z) and t. Once the velocity field 
is known, the pressure field could be determined from it by employing the Navier- 
Stokes equation [1], A technique for measuring the pressure variable p(x,y,z) was 
described in ref. [2], 

The earliest work towards the noninvasive measurement of flow by nuclear 
magnetic resonance (NMR) in vivo was performed by Singer [3]. In this tech- 
nique, a bolus of liquid is tagged upstream and detected downstream. The measure- 
ment of flow velocity consists of noting the time interval between pulsing the 
transmitter and receiving the signal, then dividing the distance between the trans- 
mitter and the receiver by the time interval. Such a technique is limited to measur- 
• ing the average flow velocity in a known direction, and has been limited in practice 
to the measurement of flow in inanimate objects or to the measurement of flow in 
the extremities. 

In 1983, Moran [4] described a new method for the reconstruction of NMR 
flow images. In this technique, an NMR scan is specifically modulated to be spec- 
troscopically motion sensitized. At the end of data collection, a special reconstruc- 
tion algorithm yields an NMR image whose pixel values are the joint spatial-flow 
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density of spins within each pixel. The advantage of flow imaging over other 
techniques, which are not able to provide spatially resolved information, is that it 
does not require any special calibrations and could provide flow measurements 
within each pixel. 

The generalized Bloch's equation in the presence of flow will be presented in 
Section 2. Examples of incoherent flow field characterization will be given in 
Section 3. The conclusion is in Section 4. 

2. Bloch's Equation with Flow 

Bloch's equation [5] describes the evolution of magnetization density in the 
presence of magnetic fields. It also takes into account various relaxation processes 
empirically. If there exists a flow field within the object in addition to the station- 
ary parts, the Bloch's equation can be modified to take the effects of flow into ac- 
count. In a Lagrangian reference frame which is rotating at the center Larmor fre- 
quency too = Y Bo and is also translating with a moving group of nuclei, the 
modified Bloch's equation for the transverse magnetization is, 

^ = i yG(t) • [r + Jv (r; f)df ]M -i£ (1) 



where r 0 is the initial position of a moving group of nuclei at time t=0. In equa- 
tion (1), v(r 0 ;t) and M = M(r 0 ;t) are the time dependent velocity vector and the 
complex transverse magnetization of the moving group of nuclei labelled at an 
initial position r 0 . In the above equation y and T2 are the gyromagnetic ratio and 
spin-spin relaxation constants, respectively. In equation (1), G(t) is the time de- 
pendent magnetic field gradient used to encode various parameters of interest The 
effect of the motion of the nuclei is absorbed into the square bracket in eq. (1) 
which represents the time dependent instantaneous position of the moving group 
of nuclei. The scalar product gives the magnetic field strength experienced by the 
group of nuclei while in motion. The solution of eq. (1) is given by, 

M(r Q ; t) = M(r o ; 0)exp{iy J G(t') • [r + J v(r Q ; t")dt"]dt'- ^} (2) 

00 2 

Equations (1) and (2) are useful when flow or motion effects of the nuclei are being 
studied. It should be noted that the flow field under consideration here is laminar. If 
one is dealing with turbulent flow, the situation is more complicated due to the 
rapid and random motion of each spin. In a later section, we will show how the 
effect of turbulance could be taken into account 

We will now outline how the solution of eq. (1) given in eq. (2) could be used 
to spatially map various flow quantities. In this article, we will follow the work of 
Moran [4] who introduced a function A(r,v) called the joint spatial-flow spin den- 
sity distribution. The multi dimensional function A gives the probability of a spin 
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to be within the location r to r+dr with a velocity in the range v to v+dv. Moran 
also proposed methods for obtaining the six dimensional quantity A(r,v) using 6- 
D imaging. The basic idea is described below. 

Since each spin can be in motion, its position may be written as 

i 

r(t) = r 0 + Jv(r 0 ;t')dt' (3) 
o 

For constant velocity, equation (3) simplifies into 

r(t) = r 0 + v t (4) 

We will now define two vector qualities given by 

t 

q=?f GCt^dt 1 (5. a) 
o 
t 

p = Y jt'CtDdt 1 (5.b) 
o 

The detected signal from a volume excited by an r.f. pulse is given by, 
S(q, p) = J J A(r o , v)exp[i(q t q + p* v)]dr Q dv (6) 

A complete data set S(q,p) covering the whole q-p space can be obtained by per- 
forming a 6-D scan. After a 6-D inverse Fourier transform A(r,v) can be 
calculated. 

In order to vary q and p independently, one has to devise special gradient 
waveforms as discussed in reference [6] by Xiang and Nalcioglu. For example, if 
we utilize the two gradient waveforms shown in fig. (1), we can manipulate q and 
p independently for the case of constant flow. If the flow is non-stationary, one 
can devise even more sophisticated gradients, see rcf [6]. 

3. Incoherent Flow Field Characterization 

Flow, in general, can be categorized as steady or time dependent. Both of these 
types of flow can then be subdivided into coherent and incoherent How. The main 
difference between coherent and incoherent flow is that in the latter there exists 
randomness either space or time or both. Whereas the former is subjected to 
"normal" statistical uncertainties associated with any physical measurement. Table 
1 summarizes various types of flow according to the classification introduced. 

In the next sections we will discuss several different examples of incoherent 
flow field characterization. The specific types of flow to be considered are I.B.b, 
U.B.a, and II. Bx according to the classification given in Table 1. 
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Figure 1 (a) Pure spatial encoding gradient, (b) pure velocity encoding gradient. 
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Table 1. Types of flow 
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3,a Piffasipn 

Diffusion is an irreversible process which is random both in time and space. It 
corresponds to zero net flow but a certain amount of transverse magnetization de- 
phasing if magnetic field gradients are present. Thus it results in a reduction of the 
measured signal amplitude. The effect of self-diffusion can be introduced into the 
Bloch's equation(l) by adding a term, 



(7) 



where Z>is the self-diffusion tensor. In the case of scalar self-diffusion, equation(7) 
may be simplified into, 



V 2 M 



(8) 



A method for the measurement of spin diffusion was introduced by 
Stejskal[7]. His technique was not based on measurements from images, i.e. no 
spatial maps of D could be obtained. Spatially resolved measurement of diffusion 
constant was proposed by various researchers[8-ll], A serious drawback of such 
techniques is that other types of motions corrupt the measurements since they may 
also cause a certain amount of spin dephasing. It can be shown [12] that the 
effective diffusion constant, D*, computed from magnitude images in the presence 
of coherent bulk flow and incoherent motion is given by, 

D*= D--^ln|a-f)+f n(p)exp[27cip*<v >]| (9) 

where f is the fraction of spins which are stationary, <v> average bulk flow 
velocity at a given point, and Ck and IT(p) are defined by, 



j = l 
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.0 1 


* 



(10. a) 



and 



Il(p) = Jd uii(u)exp [- 2icip* u] (10. b) 

where j\ is the distribution function for incoherent flow velocity u. The pulse se- 
quence given in ref.[12] was designed to minimize the contributions from other 
types of flow by employing motion compensated gradients. 



3.b Spatially Random Directional Steady Laminar Row 



There are certain types of media which are comprised of an ensemble of ran- 
dom directional flow channels. Examples of such media include porous materials, 
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and certain types of biological tissues filled with capillaries or tortuous microves- 
sels. Even though flow in each channel may be steady and laminar, because of the 
randomness of the flow channels one may still have a finite amount of dephasing 
of the transverse magnetization. This is due to the motion in flow channels when 
magnetic field gradients are present. In references[13-14] we have introduced a 
method for characterizing flow within such media by measuring a quantity called 
the incoherent diffusivity Di(r). It is defined by, 

^«=t^(r)/8 (11) 

where c u 2 (r) is the variance of the random directional flow velocity u as was dis- 
cussed in Section 3.a within a different context In equation(l 1) t is the duration of 
the gradient waveform applied for the measurement. Several applications of the 
technique were presented in refs.[15-16]. Some of the assumptions used in refs.[13- 
16] were critically examined in ref.[17]. There we have considered the effect of the 
channel length, average channel flow velocity, and the data acquisition time on the 
signal amplitude. Since the measurement of incoherent diffusivity is based on sig- 
nal attenuation due to random channel dephasing a clear understanding of the inter- 
relation these variables is crucial for accurate measurements. 

3. c Turbulent Flow 

Magnetic resonance imaging offers a means for the spatially resolved charac- 
terization of turbulent flow. When one deals with turbulent flow the measured sig- 
nal is attenuated due to the random nature of turbulent flow. In turbulent flow the 
flow field is random both in space and time. We have previously reported[18-19] 
that the Bloch's equation can be modified to take into account the turbulent nature 
of flow. This was achieved by the introduction of eddy diffusivity, Dt, into the 
Bloch's equation. In this case an additional term given by, 

D T V 2 M (12) 

was included in equation(l). The measurement of Dj is based upon the acquisition 
of two images the first one being allowed to be affected by turbulent flow whereas 
a second one in which its affects are minimized. A knowledge of the various 
imaging parameters which enter into the measurement allows one to calculate D? 
on a pixel by pixel basis. Once the eddy diffusivity is measured one can also study 
the Lagrangian integral time scale, Tl, defined by, 

D T = <u 2 >T L (13) 

4, Condition 

We have classified various types of flow according to their time dependence 
and randomness. Three different types of incoherent flow were characterized by a 
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parameter called the diffusivity. In the case of molecular diffusion it is known as 
molecular diffusivity. Spatially random laminar flow and turbulent flow were also 
described in terms of their respective diffusivities. By introducing these terms into 
the Bloch's equation we were able to show how they may be measured as a func- 
tion of spatial position by magnetic resonance imaging. 
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Pneumothoraces are clinically important abnormalities 
that appear commonly as a subtle, fine line pattern on chest 
radiographs. Few investigators have reported about 
detection of pneumothorax using computer. In 1963, Meyers 
and his colleagues reported that a pneumothorax was 
rendered clearly visible on a video display by using the 
"derivative function" image. This paper reports an 
approach for automated detection of pneumothorax to aid 
radiologists* diagnosis. 

The chest images used here were digitized by a Konica 
laser scanner. The pixel size is 0.175mm x 0.175mm. The 
matrix size and gray scale are 2000 x 2430 and 10 bits, 
respectively. The lung regions were identified by analysis 
of the horizontal signature and the edge gradient to examine 
the apical areas selection where subtle pneumothoraces 
commonly appear. To enhance a subtle pneumothorax 
pattern, we determined the orientation of the maximum edge 
gradient, which is obtained by using an edge filter in both 
vertical and horizontal directions in the digitized images. 
With this processing, the rib edges are also enhanced. 
Therefore, to distinguish a pneumothorax from rib edges, we 
detect the ribs with a technique based on statistical analysis 
of the edge gradients and their orientations that are 
included in a small region. Points located on a line which is 
suspected to be a pneumothorax in apical lung area of this 
enhanced image, were then detected using a Hough 
transform. 

When a limited angle toward the upper peripheral 
region is selected for displaying maximum edge gradients in 
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chest images, apical pneumothoraces and upper margins of 
posterior ribs were clearly visualized. It was possible to 
detect pneumothoraces by an appropriate choice of angles 
and radiuses in the parameter space of the Hough transform. 
The detected line corresponding to a pneumothorax was 
displayed on the digitized chest image. 

According to our analysis of pneumothorax patterns in 
chest images, the edges of pneumothorax have slightly 
larger gradient than that of background trend. However, 
the gradient of the edges of pneumothorax is toward upper 
peripheral side, whereas that of the background trend is 
toward the midlung side. Therefore, an appropriate 
selection of the orientation of edge gradient makes it 
possible to detect pneumothorax, even when it is subtle. 
Since the edge of typical pneumothorax lies continuously on 
a slightly curved line, a Hough transform is useful for its 
detection. 

We believe that this technique has the potential to be a 
valuable aid to radiologists for the detection of 
pneumothorax. 

Reference 

Meyers, P.H., Becker, H.C., Sweeney, J.W., Nice, CM., 
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MR Angiography (MRA) has shown considerable 
potential in non-invasive evaluation of the 
vascular system. To date this has been 
primarily confined to evaluation of the carotid 
bifurcation. The aim of this paper is to show 
the general applicability of MRA to vascular 
structures throughout the body. 

TECHNIQUE 

We utilized 3-dimensional volume acquisition of 
a tissue slab. The slab orientation, number 
and thickness of slices, and thus slab volume 
were user selectable. The sequence 

incorporated choice of presaturation band 
selection whether in or out of the plane of 
section as well as orthogonal to that plane. 
We thus had the choice of: 

a) no presaturation; 

b) one saturation band, or; 

c) a sandwich of two saturation bands. 
The use of two saturation sequences permitted 
us to acquire data and utilize the minimum 
basis set angiographic technique (MBS -MRA) as 
discussed by Sattin (1) . This has the ability to 
display either arterial or venous systems or a 
combination of both. 

The pulse sequence employed a modified field- 
echo (FE) with user selectable TR in the range 
45 to 85 msec, TE non-selectable = 10 msec, and 
selectable flip angle, usually 25°. Slice 
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thickness varied from 0.6mm to 2.0mm yielding a 
slab 3.2 cm to 12.8cm. thick. Acquisition 
times ranged from 6.6 to 13.2 minutes. Minimum 
field of view was 25 cm. 

All data was acquired on a Picker Vista* 1.5 T 
imaging system using a linear body coil or a 
quadrature head coil (for carotid or small' body 
parts) . 

COMPUTATION 

Images were reconstructed during data 
acquisition so that 2-D cross-sections were 
available for review, commencing immediately 
after scan completion. In this format the 
images are inaccessible to diagnostic 
interpretation and require minimal additional 
processing to permit projective display of the 
selected vascular component, in any plane. 
Algorithm manipulation of the data allowed 
visualization of either the arterial or venous 
systems . 

The above manipulation was performed on either 
the scanner hardware or by tape transfer on the 
VoxelFlinger** workstation. Appropriate 
rendering of data was available on each system, 
with interactive display available on the 
VoxelFlinger. 

METHODS 

Images were acquired from normal volunteers and 
patients with known vascular disease who had 
been studied by conventional angiography or 
Doppler ultrasonography. Regions studied 
included carotid bifurcation, aortic arch 
and adjacent great vessels, abdominal aorta and 



* Picker International, Highland Hts, OH 4414 3 
** Reality Imaging, Solon, Ohio 44139 
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renal arteries, aortic bifurcation and iliac 
system, femoral artery, wrist, ankle and knee. 
To date we have examined 24 subjects. 

RESULTS AND CONCLUSIONS 

The study is ongoing and results at this stage 
are preliminary. The main conclusions to date 
are: 

a) satisfactory images of arterial and 
venous systems can be obtained by 
this method. 

b) careful attention to scan technique, 
especially with regard to siting of 
the presaturation band(s) is 
necessary. 

c) it is possible to visualize small 
third-order branches subsequent to 
the image processing techniques 
described above. 

d) interactive display enhances 
visualization of small vascular 
structures, and allows clear viewing 
of deeper structures without 
obscuration by overlying tissue. 

e) the minimum basis set method allows 
complete recognition of arterial and 
venous morphology. 
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FIGURE 1: THORAX 

a) 45° oblique coronal section demonstrates pulmonary artery bifurcation, 
azygos system, and internal mammary arteries, b) 20° RAO axial section 
demonstrates azygos system and vertebral venous plexus. Superior vena cava 
is not seen due to superior placement of presaturation band. Azygos system 
is visible because of flow direction and is thus unaffected by 
presaturation band. 




FIGURE 2: AORTIC ARCH 

a,b,c) Arterial phase with progressive rotation to 45° oblique. The arch is 
displayed in transverse projective view in (d) which shows take-off of 
innominate artery and its bifurcation. 



FIGURE 3: RENAL VENOUS AND ARTERIAL SYSTEMS 

a,b) Axial sections with presaturation bands above (vein) and below (artery) 
plane of section. Note absent signal from aorta and renal arteries in (a) 
and similarly absent signal from inferior vena cava in (b). 




FIGURE 4: ILIAC ARTERY 

Arterial phase coronal (a) and rotated coronal (b,c) images of iliac artery 
system in patient with Large uterine Leiomyoma. Image (d) is transverse 
projective display showing iliac and gluteal arteries. 
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FIGURE 5: FEMORAL ARTERY 

a,b) Femoral arteries with inferior presaturat ion band. Mote distracting 
background noise in non- subtracted image (a). Minimum basis subtraction 
set technique (b) allows enhancement of femoral arteries in relation to 
surround. Note artifact in upper field in (b) due to current algorithm. 
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FIGURE 6: WRIST 

a, b) Arterial phase coronal projections of wrist showing radial and ulnar 
arteries and transverse palmar arch. Note ability to visualize small 
caliber vessels using subtraction technique. 
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Fractures of the tibial plateau are common 
injuries(l,2,3,4) . Conventional radiography, 
including Computerized Tomography, may be inadequate 
for evaluation of these fractures due to the complex 
anatomy of the proximal tibia(5,6, 7) . Interactive 
3-D manipulation overcomes this limitation and 
allows optimal visualization of the tibial plateaus. 
We report our initial experience in diagnosis and 
assessment of tibial plateau fractures using the 
VoxelFlinger* workstation for interactive 3-D 
viewing . 

MATERIALS AND METHODS 

Computed Tomography (CT) was performed in 10 
patients with tibial plateau fractures using a 
Picker 1200 CT Scanner**. Images were acquired with 
a 512 x 512 matrix and bone reconstruction 
algorithm. Contiguous 2mm thick transaxial images 
were acquired from the femoral condyles through the 
tibial plateaus in 9 patients, and contiguous 3mm 
thick transaxial images in one patient. The 2-D 
images were transferred via magnetic tape to the 
VoxelFlinger workstation for interactive 3-D 
viewing . 

We assessed the utility of interactive 
manipulation according to the following criteria: 



* Reality Imaging, Solon, Ohio 44139 
** Picker International, Highland Hts., Ohio 44143 
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1) Confirmation of fracture 

2) Ease of visualization of fracture lines 

3) Degree of impaction and displacement 

4) Extent of expansion (explosion) of the 
plateau. 

RESULTS 

The data could be transferred to the 
workstation, reviewed and imaged within 30-45 
minutes. Once reviewed, it became apparent 
that the 3-D images better characterized the 
fractures as compared to the transaxial CT images 
and orthogonal multiplanar reconstruction (MPR) . 
The 3-D images were found to be an advantage in 
visualizing sloping and irregular plateau 
depressions and impactions (Figure 1) . Oblique 
fractures could be rotated for optimal visualization 
and subtle abnormalities such as minimal plateau 
depressions were easily seen (Figure 2) . 

The fractures were classified from the CT 
images according to Hohl(8) (Table I). The data 
from the 3-D images resulted in a change of 
classification in one case, from split depression to 
comminuted upper tibia (bi-plateau) . 

Table II summarizes the degree of plateau 
depression and fragment separation as measured from 
the CT and VoxelFlinger images. Although the actual 
measurements differed by only a small amount (3mm or 
less) , the 3-D images were more accurate in 
depicting the maximum amount of plateau depression. 

DISCUSSION 

Fractures of the tibial plateau are difficult 
to assess radiographically due to the complex 
anatomy of the proximal tibia(5,6,7) . Open 
reduction is usually performed if the plateau is 
depressed 5mm or greater (8). Other factors which 
require assessment for determining treatment include 
the degree of fragment separation and comminution. 
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Accurate evaluation is essential in order to 
maintain adequate range of motion and to prevent 
joint instability and early degenerative change(6). 

CT has largely replaced conventional tomography 
in the evaluation of tibial plateau fractures (5, 6) . 
However, the transaxial sections obtained by CT may 
not be accurate for assessment because of the slope 
of the plateau(9). In addition, because the 
fractures are often oblique, neither transaxial 
sections nor coronal and sagittal reconstructions 
are capable of displaying them adequately. 

Interactive 3-D viewing lends itself well to 
the evaluation of tibial plateau fractures • The 
image can be manipulated so that the fracture can be 
seen in its most optimal projection. The 3-D format 
allows for excellent characterization of plateau 
depression and expansion. 

Several techniques were valuable in fracture 
assessment. In the interactive MPR mode, 3 
orthogonal planes could be viewed and manipulated 
simultaneously. This allowed the relationships 
between complex fractures, depressions and fracture 
fragments to be assessed immediately (Figure 3) . By 
using the "Cut Plane" and "System Parameter" (TRIM) 
modes, overlying and surrounding structures such as 
casts and femoral condyles could be removed, 
allowing better visualization of the plateau: for 
example, removal of the condyles by this method 
permitted unobstructed en face viewing of the 
plateau surface from above. 

The 3-D images could be manipulated 
interactively for optimal viewing as opposed to 2-D 
CT reconstructions, where the planes were preset. 
The degree of plateau depression and fragment* 
separation could be measured directly, allowing for 
more accurate measurements. As a Level I Trauma 
Center, the majority of cases at our hospital 
resulted from severe injuries and open reduction was 
necessary. With less severe injuries, accurate 
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measurements are essential in helping to determine 
the method of treatment. 

In summary, the VoxelFlinger allows easy 
interactive manipulation of images for optimal 
display of fractures and morphology. Overlying and 
surrounding structures, e.g., casts, bandages and 
femoral condyles can be removed from the image where 
they would otherwise interfere. Fractures can be 
seen in multiple planes and the plateaus can be 
visualized en face. In many cases, this provided 
additional information which permitted complete 
fracture characterization. 
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Figure 1 a,b: LATERAL PLATEAU FRACTURE: Coronal CT reconstruction (a) and 
3-D image (b), showing relationship of plateau depression to 
fragment lines. Note removal of condyles except for small 
fragment medial Ly. 



Figure 2a, b: 3-D image showing minimal depression of posterior aspect of 
Lateral plateau (arrow) not appreciated on original CT 
sections or reconstruction (b). Note extension fractures 
involving tibial spines. 




U JllJ CT 



Figure 3: Multiplanar Reconstruction screen showing simultaneous 
images and relationship between plateau depression, fracture 
and fracture fragment. Note that as selected plane of 
section is varied on any of these orthogonal projections, it 
automatically changes to show the corresponding planes in 
each of the other projections - and does so interactively. 
(Same Case as Figure 1). 
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Table I : Classification of Tibial 


Plateau Fractures 


TYPE OF FRACTURE 


METHOD OF EVALUATION 




CT 


VOXELFLINGER 


Central Depression 


1 


1 


Split Depression 


5 


4 


Comminuted Upper Tibia 






(Bi-Plateau) 


4 


5 



TABLE II: MEASUREMENTS OBTAINED BY CT vs. VOXELFLINGER 
(ALL MEASUREMENTS IN MILLIMETERS) 


FRACTURE TYPE 




METHOD OF EVALUATION 






C T 


VOXELFLINGER 




PLATEAU 
DEPRESSION 


FRAGMENT 
SEPARATION 


PLATEAU 
DEPRESSION 


FRAGMENT 
SEPARATION 


Central Depression* 


<2 


* I 


3 


<2 


Split Depression 


5 


2 { 


5 


2 


Split Depression 


12 


4 | 


12.5 


5.5 


Split Depression 


10 


7 ! 


10.5 


6.0 


Split Depression 


12 


5 | 


12 


5 


Comminuted*"*" 
Upper Tibia 


16 


:: 
{ 

5 i 


19 


5 


Comminuted 
Upper Tibia 


5 


> 

4 \ 

\ 


5.5 


3.5 


Comminuted 
Upper Tibia 


14 


7 ! 


14.5 


6.5 


Comminuted 
Upper Tibia 


15 


1 I 


15 


: 

i 1 

i 
i 


Comminuted 
Upper Tibia 


12 


1 | 


12 


1 

i 


+ Voxel F linger showed subtle depression not appreciated on CT 




^Classification changed from Split Depression based on VoxelFlinger data 
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1. Introduction. Contrast in an imaging acquisition is defined in terms of 
the magnitude of the signal value difference between two neighboring regions. 
Thus, the two quantities of paramount importance in image processing, con- 
trast and resolution, seem to be intrinsically local in nature. Reflecting this 
are the many local image enhancement techniques involving convolutions, 
typically effected by repeated local application of templates which are trans- 
lated throughout the image. But there is also a collection of global methods 
defined independently of the immediate spacial relationships between voxels, 
a prototypical example being histogram equalization. More generally, seg- 
mentation (which is the forced classification of pixels according to decision 
algorithms applied to their totality of characteristics) may employ both local 
and global information. Our intention here is to examine the implied "global" 
aspects of contrast. This will ultimately suggest generalizations of the cur- 
rent notions of the contrast inherent in an imaging acquisition, with various 
implications for the analysis of image quality, and for the creation of new 
classes of images from these acquisitions. Specifically, a new image process- 
ing algorithm is identified, which creates contrast from textural distinctions 
between tissues using global determinants of texture. Critical features of this 
approach are the recognition of "metric" and "non-metric" determinants of 
acquisition contrast, and the avoidance of ad hoc application of templates or 
probabilistically based decision processes. 

It should be mentioned at the outset that a medical imaging technology 
capable of optimally taking advantage of this methodology does not neces- 
sarily presently exist (though ultrasound and MM seem likely candidates for 
its application). At the moment, we believe our investigation to be of interest 
primarily for its mathematical conclusions regarding the quantification and 
presentation of the useful imaging content in an acquisition. 

2. Global Contrast. The principal aim of medical image production is 
to create distinctions between tissues. Once an object is distinguishable, it 
is of less importance whether it is "well" distinguished or "superbly" distin- 
guished. From a global standpoint, the more distinct tissues identified the 
better - and an image which adequately distinguishes ten tissues is generally 
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better than one which distinguishes two of them ten times as well as any pair 
of tissues in the first image, while failing to delineate the other eight tissues. 
Therefore, we desire to generalize the definition of contrast to be more reflec- 
tive of the number of "adequately" distinguished tissues. For example, one 
rejected candidate for Global Contrast would be the standard deviation of 
the image signal values divided by the standard deviation of the noise (e.g. 
as used in [1]) - since this strongly favors large well distinguished tissues and 
de-emphasizes the importance of small adequately distinguished tissues. 

We begin with the supposition that similar tissues map to the "same" 
location in signal space (signal space being the continuum representing the 
possible signal values which may be measured; it may be multi-dimensional 
e.g., in MM it could be the set of ordered triplets representing T1-,T2-, and 
proton density- weighted signal values). If there is no noise, and two tissues 
are the same if and only if they have the same signal value, then 

number of distinct tissues = -77 , ,.}, — rr = 1. (1) 

N ^ h(M(wi)) ^ v ' 

where the first sum is over all the voxels W{ which partition the field of 
view V, M(w{) is the point in signal space S assigned to voxel w,*, h is the 
histogram of the acquisition normalized to the N voxels in the field of view 
(i.e. h is the probability density function of the acquisition), and the second 
sum is over all points Sj in signal space such that there is a voxel wi for which 
Sj — M(wi). In the setting of noise (1) can be generalized using the change 
of variable formula (see [2]) 

J fogd\ A = J f.[4L]d\ B , (2) 

(e.g. A is V, B is 5, / = 9 = M, d/i/dA = h) so that "Global Contrast" 
(GC) can be defined as 

In (3) it is understood that h is the normalized histogram convolved with 
a Gaussian density whose standard deviation is some multiple t(v) of the 
noise standard deviation u (where it is assumed, as in MRI, that v is uniform 
throughout the image), so that S e („) is the volume in signal space having a 
non-negligible probability density of being mapped to by some voxel in the 
field of view (non-negligible as gauged by In particular, the "number 

of distinct tissues" in (1) generalizes to the noise normalized volume in signal 
space accessed (the middle expression in (3), which generalizes the sum on 
the right side of (1)). Without the decision making algorithms of segmenta- 
tion, one cannot do better. However, it is our contention that such decision 



513 



processes belong to specific engineering appplications and not to basic anal- 
ysis of contrast or information, since segmentation decisions are always in 
danger of being wrong. 

It should be noted that, unlike the signal-to-noise ratio, there is no simple 
inverse relationship between GC and resolution. For this reason one could 
propose that resolution be chosen such that GC is optimized. [For example, 
consider two tissues arranged in a checkerboard fashion. With constant dose 
or imaging time, at either very low or very high resolution GC will tend to 1 
(meaning that only one tissue is identified). Resolution should be chosen so 
that GC is close to 2.] Also, note that as the noise standard deviation tends 
to zero, GC reduces to (1), while as the noise standard deviation becomes 
large, GC becomes 1. In this sense GC is a true generalization of the number 
of distinct tissues identified. 

At this point it is of interest to compare (3) with the measurement of 
information content given by information theory [3], namely (ignoring noise) 

= ^E^t^^] = -E^)iog 2 ^). (4) 

Wi Sj 

HE refers to "histogram entropy** , and it is the classical information content 
embodied in the assignment of a collection of voxels {wi} to the signal val- 
ues {M(wi)}. The appeal of GC is that it is independent of relative tissue 
volume, whereas HE is very dependent on the relative volumes of tissues. 
Since we are concerned with rewarding that image which identifies the most 
tissues (regardless of their volume), GC would seem to have some advan- 
tage in its economy of dependencies (nevertheless, the similarity of (3) and 
(4) is striking). GC is suitable for addressing questions concerning optimal 
selection of parameters controlling an acquisition (e.g. in reference to MRI, 
relevant questions addressed would be: "when should imaging matrix size 
be traded for increased number of excitations?," or, "which TE is best for a 
T2-weighted sequence?,** etc.). 

3. Generalized Images. Medical images are typically a graphical depic- 
tion of the properties of a set of voxels. Usually these properties are phys- 
ical parameters of the tissue. However, they could also be abstract proper- 
ties of the voxels. In particular, each voxel w has the information content 
— log 2 h(M(w)). If we image this information content by assigning each u>< 
the new value — log 2 h(M(wi)) we obtain an image in which contrast is 
independent of the distance between the signal values given the voxels by the 
acquisition - a non-metric image, based on non-metric contrast (as opposed 
to the standard images, this new image is independent of the signal space 
metric). We can accomplish the same thing if we eliminate the logarithm 
and image h directly (the log does not impact the information content of the 
image). We can actually do somewhat more. Since t(v) (which defines h) 
is unspecified, we can produce a continuum of images, each with a different 
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value of These images are referred to as MT-images ("MT", stands 

for "measure-theoretic", and is used for reasons considered elsewhere [2,4]). 
Thus different MT-images are produced by filtering in signal space rather than 
filtering in the field of view space. 

We now further explore some properties of these new images: 

DEFINITION 1: A tissue is some specified collection of voxels. 

Consider the random variable which is the measured signal value dif- 
ference between two randomly chosen voxels, one from each of two different 
tissues. The contrast-to-noise ratio is the expectation of this random variable 
divided by its standard deviation - in other words 

DEFINITION 2: The contrast-to-noise ratio (CNR) of two tissues is given 
by 

CNR= * * 2 , (5) 
V^vari + var2 

where van, var2, denote the variance of the signal values of the two tissues, 
and si, 52 are the mean signal values of the two tissues. 

DEFINITION 3: A standard image from an acquisition is one in which CNR 
between two tissues is dependent on the difference in the distances from a 
fixed point in signal space that the signal values associated with two tissues 
have. 

For example, the image created by the orginal measured signal values 
is a standard image (the fixed point being the origin). It is interesting that 
in a certain limit, MT-imaging also produces a standard image, and in this 
sense MT-imaging is a more general means of producing images from an 
acquisition. 

THEOREM 1: Standard imaging is a special case of MT-imaging. 
In particular, in the limit as e(v) — ► oo, CNR in the MT-image is 

c ^ = tf.!NLdM (6) 

v2j/ 

where ||si|| and ||s2|| represent the distances from s\ and 52 to the 
mean signal value of the acquisition, and K is the ratio of the 
arithmetic mean to the root means square of these two values (K 
is always between 0.7 and 1). 

Theorem 1 is proven by application of Definition 2 to the definition of 
h (as the convolution of the normalized histogram with a Gaussian density 
of standard deviation c{v) ), and applying L'HospitaPs rule. An example is 
given in Figure 1. 

THEOREM 2: Two tissues having zero CNR in a standard image 
will in general have non-zero CNR in MT-images if they differ in 
the variance of their signal values. 
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An example of this is seen in Figure 2. While it is possible to visually 
identify the embedded tissues in the standard image using a tiny window, 
this precludes viewing the general tissue environment (e.g. seeing the other 
tissue). Further, a large CNR increase is effected in the MT-image. Note also 
that histogram equalization is not particularly efficacious in this example. 

4. Texture and Contrast. MT-imaging is a mechanism for converting 
texture into contrast. As in Figure 2, it may certainly occur that distinct 
tissues will not be recognized as such if their mean signal values are very 
close (or even identical) and their textural variations are small compared to 
the dynamic range of the image. The MT-image then converts their higher 
statistical properties into true contrast - and this is accomplished without 
mechanisms which may contaminate the data (and which may be unreli- 
able), such as those characterizing segmentation. [We do not mean to slight 
segmentation - we only argue that segmentation decisions are not inherent 
in the data, while MT-imaging is.] Neither is recourse made to texture tem- 
plates which contain various ad hoc properties (e.g. their spacial extent, and 
their particular matrix values). 

We now will examine some subtleties of the relationship of non-metric 
contrast and texture. 

DEFINITION 4: A texture is the collection of statistical properties of the 
measured signal values of a tissue. 

DEFINITION 5: For each of two tissues, subtract the mean signal value of 
that tissue from each voxel in that tissue. First order NMC is the CNR in the 
MT-image formed from the new signal values of these two tissues. Second 
order NMC is the CNR of the two tissues in the MT-image of this MT-image, 
and so on recursively for the nth order NMC. 

We will refer to first order NMC in the following as just NMC. Also, 
we will assume below that there are sufficiently many voxels in a tissue 
to adequately reflect the noise properities of the acquisition, and that the 
MT-images referred to below are those obtained directly from the histogram 
without any convolution with a Gaussian. 

DEFINITION 6: A tissue having a linear texture is a collection^ of voxels 
whose signal values are distributed on a finite interval in signal space such 
that the number of voxels having a particular signal value in this interval 
varies linearly with its absolute distance from the mean signal value of the 
tissue. If the slope of this distribution is zero (so that the signal values are 
uniformly distributed on the interval), then the texture is termed white. If 
the slope is negative then the texture is termed linear off-white. A tissue 
having a Gaussian texture is a collection of voxels whose signal values are 
normally distributed (this texture is also off-white). 

Since the metric doesn't contribute to NMC, we should be interested in 
what it actually does depend on. It would seem to be obvious that, since MT- 
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signal value is determined by the histogram of the signal values, NMC must 
be heavily dependent on the difference in volume between the two tissues. 
Therefore, the following theorem is non-intuitive and surprising: 

THEOREM 3: Let the number of voxels in each of two tissues be 
rrix and m 2 . Let crj < <j 2 be the widths of the domains in signal 
space for which the histograms of each tissue are non-zero for lin- 
ear textures, or represent their respective standard deviations for 
Gaussian textures. Then 

1) For two tissues with white texture, 



In particular, NMC is independent of the relative volumes of the 
two tissues, and increases without bound as 02/01 — ► oo. 

2) For two tissues with linear off-white textures, 
a. if 02/01 » ™>2l™\<i then as (02/01) becomes large NMC tends to 



b. if 02/01 << m 2 /mi, then as 02/01 becomes large NMC tends to 



c. for 02/01 close to 1, NMC ~ (02/01 — 1)- 

3) For two tissues with Gaussian texture 

a. if 02/01 » m2/mi, then as 02/01 becomes large NMC tends to 
(2/V3-l)~ 1 / 2 «2.54, 

b. if 02/01 « "12/mi, then as 02/01 becomes large NMC tends to 
(y/2- 1) • (2/V3 - l)' 1 ' 2 « L05, 

c. for 02/01 close to 1, NMC ~ fc(<r 2 M - 1), where * = [1/(2^)] • 
(2/v^-l)- 1 / 2 «0.90- 

This theorem is proven by direct substitution of the texture definitions 
into the equations for CNR and NMC as given in definitions 2, 5, and 6. 
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FIGURE 1. A is a proton density weighted MR image of a normal brain. 
B is the MT- image with c(u) = v. C is the MT- image with e(^) = 30i/. 
Though windowed a little differently, A and C are nearly indistinguishable 
in terms of Grey matter/White matter CNR (2.80 versus 273 in A and C, 
respectively). 




FIGURE 2. A is a numerical phantom. The left side of the image has mean 
signal value 200, while the right side has mean signal value 100. Gaussian 
noise of standard deviation 1 has been added throughout, except that there 
is a small region in the upper left and lower right having standard deviation 
2, these representing tissues distinct from their background. In a standard 
image (e.g. A) the only way an embedded tissue can be seen is with a window 
width near 1, precluding visualization of the general tissue environment (e.g. 
the tissue in the lower right). CNR for the embedded tissues and their 
background is nominally zero (measured with ROIs to be 0.03 for the left 
sided tissue pair). B is the MT- image made from the signal values in A with 
e(f) = v. CNR for the left sided tissue pair is 0.59. C is the histogram 
equalization of the original signal values of the phantom. CNR for the left 
sided tissue pair is measured to be 0.14. 



I 



Tomographic Reconstruction 
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ABSTRACT 

MR Imaging studies of 33 patients with various 
disorders of the musculoskeletal system were 
reconstructed using surface and volume rendering 
techniques. Surface reconstruction provided a more 
accurate display of the pathoanatomy than volume 
reconstruction when multiple objects with anatomic 
overlap had to be demonstrated. Volume reconstruction 
was characterized by greater detail than surface 
reconstruction in demonstrating fine anatomic structures. 
Both methods were limited by the need for disarticulation 
lines surrounding the object in the 2-D images to 
exclude other structures with signal intensity similar to 
the object from the 3-D display. 

INTRODUCTION 

Magnetic resonance (MR) imaging has been established as 
an important diagnostic modality for evaluating the musculoskeletal 
system that supplements and often replaces computed tomography 
(CT). Despite its multiplanar imaging capability, MR imaging is 
currently limited by a two-dimensional display of frequently rather 
complex three-dimensional structures. Conception of the spatial 
configuration of an object therefore requires mental integration of a 
series of two-dimensional MR images. Although radiologists are 
accustomed to this type of image analysis, it may pose a difficult 
problem for the clinician who is treating the patient. 

In CT, three-dimensional (3-D) reconstruction of a series of 
two-dimensional scans has already been widely used for 
preoperative planning (1-4). 3-D reconstruction of MR images has 
been limited due to technical problems. Only recently, 
improvements in MR acquisition and 3-D software techniques have 
facilitated 3-D reconstruction of MR images (5-10). The 3-D image 
quality that can be achieved with the different 3-D rendering 
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techniques available for generating 3-D MR image displays has, 
however, not been compared yet. 

In this study, we produced 3-D MR reconstructions in 33 
patients with various disorders of the musculoskeletal system. 
Surface and volume reconstructions were obtained in all 33 
patients. Image quality, diagnostic results and technical aspects of 
both techniques were prospectively evaluated. 

METHODS 

Thirty-three consecutive patients who had been referred to 
our institution for a musculoskeletal MR examination were selected 
for 3-D reconstruction of MR images. MR imaging studies of the hip, 
knee, ankle, shoulder or wrist were obtained in 13, 10, 4, 4, and 2 
patients, respectively. MR images were generated on a General 
Electric Signa system operating at a field strength of 1.5T. Imaging 
studies included T1 -weighted (TR=300-500 msec, TE=12-30 msec) 
and T2-weighted spinecho (TR=1 800-2500 msec, TE=80-240 
msec) as well as T2*-weighted gradient-echo (GRASS: gradient 
recalled acquisition in the steady state, TR=50-100 msec, TE=20- 

40 msec, 0=10-30°) sequences. Hip studies were produced using a 
circumferential body coil or a Helmholtz coil; knee and ankle scans 
were generated with a lower extremity coil; shoulder imaging was 
performed employing a pair of counter-rotating current-loop-gap 
resonators, and wrist images were acquired with a special wrist 
coil. FOV was kept as small as technically feasible in order to 
obtain a high spatial resolution of the MR images. Image matrix 
varied from 256x192 to 256x256. 

Two-dimensional and 3-D image processing was performed 
on an independent imaging system manufactured by Dimensional 
Medicine Inc. (Maxiview; Motorola 68020 processor; 8 MB RAM; 
766 MB hard disk; 4 MB graphics memory; 19 in - color display). 
3-D displays of the different tissues were computed on the basis of 
the T1 or the T2-weighted spinecho or the T2*-weighted gradient- 
echo images (Fig. 1). The sequence that demonstrated the best 
contrast for a given tissue was usually selected for 3-D 
reconstruction. 

When other structures had a similar signal intensity than the 
tissue in question, a manually or automatically drawn 
disarticulation line surrounding the object was used to image only 
what was desired in the 3-D display (Figs. 1, 2). Disarticulation 
lines were selected by the operator on a slice-by-slice basis. 
Because the use of a disarticulation line may result in a step-like 
outer contour of the object in the 3-D display (7), a disarticulation 
surface interconnecting pairs of upper and lower disarticulation 
lines was computed (Fig. 1). 
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The data were resampled by linear interpolation to generate 
a uniform resolution in all three dimensions (Fig. 1). The resultant 
volume consisted of cubic volume elements which formed the basis 
for all further processing. 

Three-dimensional displays were then generated using 
surface and volume rendering techniques (Fig. 1). In surface 
reconstruction, only the surfaces of the objects were computed 
using 3-D boundary detection; a voxel was considered part of the 
object when 1.) its signal intensity fell within a predefined range of 
signal intensities representative for the structure in question and 2.) 
when it was located within a disarticulation surface in case 
disarticulation lines had been added to the image. In volume 
reconstruction, all voxels not only those located at the object 
surface were preserved and displayed in the 3-D image. The grey 
value of a voxel in the 2-D image was transferred into a 
corresponding color and transparency in the 3-D volume rendition 
using an empirically defined tissue classification table. When 
disarticulation lines had been added to the 2-D images, only those 
voxels that were located within a disarticulation surface were 
displayed in the volume reconstruction. In both surface and volume 
reconstruction, the different objects (in the knee for example 
anterior and posterior cruciate ligaments, menisci, and osseous 
structures, i.e. femur, tibia, patella, and fibula) were simultaneously 
displayed using different colors and opacities. 

RESULTS AND DISCUSSION 

Surface reconstruction 

In imaging the hip, knee, ankle, shoulder and wrist, 3-D 
surface reconstruction provided sufficient anatomic information in 
most of the cases. Surface reconstruction was particularly good in 
demonstrating the spatial relationship between different, 
overlapping objects. In the knee, for example, the orientation, the 
origin and the point of insertion of the anterior (ACL) and posterior 
cruciate ligaments (PCL) were clearly delineated in all 10 patients 
(Fig.2C). 

Surface reconstruction was, however, limited when fine 
anatomic structures such as the sacroiliac (SI) joint had to be 
displayed. In imaging the pelvis, the SI joint could not be 
demonstrated in 40% of the patients. In general, fine anatomic 
detail was often obscured in surface reconstruction. In addition, 
artifactual irregularities were frequently observed along the outer 
contour of the femur, tibia, and humerus. 

Surface reconstruction requires 3-D boundary detection and 
displays only a subset of the data available in the 2-D scan, i.e. the 
object surface. Exclusion of voxels that are not considered to be 
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Ha""! 1: 

Data Processing for 3-D Reconstruction 



ORIGINAL DATA (T1, T2, T2*-WEIGHTED) 



MANUAL/AUTOMATIC 
DISARTICULATIONLINE 



DISARTICULATION- 
SURFACE 



LINEAR INTERPOLATION 



SURFACE RECONSTRUCTION 




VOLUME RECONSTRUCTION 







part of the object surface from 3-D display may cause both failure to 
demonstrate a structure such as the SI joint as well as irregularities 
in the outer object contour. 

Volume reconstruction 

Volume reconstruction was characterized by excellent 
anatomic detail. Volume reconstruction was, however, limited when 
several structures that demonstrated anatomic overlap were 
displayed simultaneously. The orientation, the origin and the 
insertion of the cruciate ligaments were difficult to evaluate in 3-D 
volume reconstructions of the knee (Fig. 2D). In 7 of the knee 
patients, the ACL incorrectly appeared to be located posterior to the 
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PCL (Fig. 2D). This was in part caused by the high transparency 
effect of the volume reconstruction. 

Fine anatomic structures such as the SI joint were, however, 
clearly delineated in 3-D volume reconstruction. 3-D image artifacts 
along the contour of the femur, tibia, and humerus were less 
frequently seen in volume reconstruction than in surface 
reconstruction. 

Volume reconstruction preserves all information contained in 
the 2-D MR images. Exclusion of voxels from 3-D display is not 
required in using volume rendering techniques. Computing time is, 
however, increased by a factor of 8 - 10 in volume reconstruction 
when compared to surface reconstruction, because significantly 
more voxels need to be rendered. 

Disarticulation 

Both surface and volume reconstruction required manually 
or automatically defined disarticulation lines surrounding the object 
in the 2-D MR images (Fig. 2B). Although MR imaging affords good 
soft-tissue contrast, many different structures frequently display 
similar signal intensity in the 2-D image. Any signal intensity based 
surface rendition or any tissue classification that transfers 2-D 
signal intensity values into different colors and transparencies in 
volume rendering would therefore include structures other than the 
object to be displayed; the resultant 3-D images would give a 
misrepresentation of the true anatomy. Therefore, disarticulation 
lines are needed that help to image only what is part of the object in 
the 3-D display (5,7). The introduction of disarticulation lines in the 

2- D images is, however, time-consuming since it is performed on a 
slice-by-slice basis; at present, it represents a major limitation for 
both 3-D surface and volume rendering of MR images. 

CONCLUSION 

In most patients, both surface and volume reconstructions 
provide a diagnostically useful 3-D display of the pathoanatomy 
that can be useful to the surgeon for preoperative planning. 
Surface reconstruction is superior to volume reconstruction when 
the spatial relationships between various objects with anatomic 
overlap need to be displayed. Volume reconstruction is better than 
surface reconstruction in demonstrating fine anatomic detail. Both 
methods are currently limited by the need for disarticulation lines 
that surround the object to be imaged. In the future, software 
routines that provide automatic definition of these disarticulation 
lines without operator interaction will greatly enhance the use of 

3- D rendering techniques for MR imaging. 
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Figure 2: 

3-D MR Surface and Volume Reconstruction 
of the Knee in a 33-vear-old Patient 





Fig. 2A: 

2-D MR image (spin echo, TR=500 msec, 
TE=20 msec). Femur, tibia, pateiia, and 
fibula demonstrate high signal intensity. 
Hoffa's fat pad (1) and soft-tissue posterior 
to the distal femur (2) display similar 
signal intensity than the osseous 
structures. 




Fig. 2B: 

Selection of voxels with signal intensities 
representative for bone for 3-D 
reconstruction of osseous structures. All 
voxels with signal intensity representative 
for bone are highlighted in blue. Since 
Hoffa's fat pad (1) and soft-tissue posterior 
to the distal femur (2) have similar signal 
intensity, they would be registered as part 
of the bony structures resulting in 3-D 
image artifacts. Consequently, disarticu- 
lation lines surrounding the bone are 
needed to exclude soft-tissue from 3-D 
display. Automatically defined disarticu- 
lation line around proximal tibia is shown 
(yellow, arrowheads). 




Fig. 2C: 

3-D MR surface reconstruction, posterior 
view. Osseous structures (white), menisci 
(magenta), anterior cruciate ligament 
(ACL) (yellow, arrowheads) and posterior 
cruciate ligament (PCL) (blue) are 
simultaneously shown. ACL is partially 
hidden by medial condyle in this 
projection. Origin and insertion of PCL is 
well shown; PCL is clearly posterior to 
ACL 



Fig. 2D: 

3-D MR volume reconstruction, posterior 
view (same projection as in C). Due to the 
transparency effect, it is not certain 
whether the ACL (yellow, arrowheads) is 
located anterior or posterior to the PCL 
(blue) in the volume reconstruction. 
Volume reconstruction Is limited when 
multiple overlapping objects are 
displayed. 
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1 . INTRODUCTION 

Digital reconstruction of image from projections has 
become a very active field of research and application 
in many different fields, especially in biology and 
medicine. Different techniques have been proposed "for 
image reconstruction from projections. The most commonly 
used methods are transform methods, series expansion 
methods. 

As image reconstruction from projections is a problem of 
estimation the original image from projection data. It 
is natural that the quality of the estimated image will 
depend upon the criterion used in estimation process. 

Almost all CT Scanners today use the single criterion 
method to estimate the original image from projections. 
But actually the quality of an image usually involves 
several conflicting and nonccmmensurable criteria. It 
is obvious that different aspects of the image field 
must be considered during reconstruction. 

In this paper we developed a multicriterion image recon- 
struction method with man-machine interaction as an aid 
in solving the multicriterion optimization problem. 

To accelerate the reconstruction procedure a pyramid 
reconstruction approach has been explored. 

2. MULTICRITERION IMAGE REC30NSTRUCTION 

There are several different criteria used in single 
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criterion reconstruction algorithm. Such as maxmium 
entropy [2 ], least square errorM], nonuniformity and 
peakness[4]. It is natural that as a starting point we 
use the combination of these criteria to develop the 
multicriterion optimization method. 

To reconstruct an image using multicriterion is equi- 
valent to solving a vector optimization problem (VOP) of 
the form: 

D1 . min g(x)=(g 1 (x) f g 2 (x) f g 3 (x) ) 

where g^ (x) is negative entropy function of the image , 

g2<x) is the sum of nonuniformity function and peakness 

function of the image, g 3 (x) is square error function 

of original test image and reconstruction, x is n- 
dimensional vector of dicision variables x. 

Upon satisfying certain conditions, the vector optimi- 
zation problem D1 can be stated mathematically in term 
of optimal solutions of appropriate scalar optimization 
problem D2 (SOP) in the following manner: 

D2. nu^nize ^(x). wTg(x) 

where w. >0 and JL 

£.,-1 

For a n-dimensional decision vector x the entropy func- 
tion is: 

9i( x )=-(-g i Xflnx f ) = jC i Xflnxi 

The sum of a nonurriformity function and peakness func- 
tion is: 

g 2 (x)=^otx T Sx + i-x T x 

where matrix |Sij| nxn is called nonuniformity matrix[4], 

a is a constant chosen experimentally to yield the best 
reconstruction. The mean square error function is: 
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9 3 ( x HP(y-A>0 T (y-Ax) 



where ft is a constant to be determined experimentally , 
y is a mnaimensional projections data vector, A is 
projection matrix. 

We proposed an interactive method to solve D2 and the 
method can formalized as follow: 
Step 1 . 

Set weight values w^ ,w 2 fW-j, starting point x, con- 
stant a f jj , and the ideal criterion values 

Step 2 

Typical iteration 

xf k+0 = P(x (k >) j=1,2,...,n, k=0. 

1 2 

Determine the two solution x and x , 
1 2 

if x =x then problem is solved, goto step 4 

if 12 

the decision-maker accept x or x as optimal 

then the problem is solved goto step 4, Otherwise 

k=k+1 goto step 3. 

Step 3 

Decision-maker questioning 

Q1:"Do you consider the g 1 (x ) satisfactory? 

Rlr'Ves", goto Q2, "no" goto (2) 

Q2:"Do you consider the g 2 (x ) satisfactory? 

R2: ,l yes n , goto Q3, "no" goto (2) 

Q3:"Do you consider the ) satisfactory? 

R3:"yes" r goto (1), "no" goto (2) 
Machine updating 

( 1 ) x is an optimal solution of D1 , goto step 5 

(2) Determine new weighting vector w^=(v£ r *£ f v£) 
k=k+1 , goto step 2. 

Step 4 Determine reconstructed image 

(i,j)=x ((n x(i-1 ))+j), i=1,...,n, j=1,...,n 
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Table 1 is the result of one of the interactive image 
reconstruction process. In this case we adjust the weigh- 
ting vector w to make three criteria of the estimated 
image match the three ideal criterion values. 

3. PYRAMID IMAGE RECONSTRUCTION APPROACH 

To acceralate the reconstruction process a pyramid app- 
roach of reconstruction has been developed. Principle 
of the method is shown in Fig.1, i.e. the original image 
is deduced to a series of low resolution images (image 
pyramid). The image reconstruction was done on the low- 
est resolution level. Then the reconstructed image was 
expanded to a higher resolution level and modified by 
the data of the original Gaussian image of the corre- 
sponding level. This process is repeated until the high- 
est resolution image is obtained. 

4. SIMULATION RESULTS AND DISCUSSION 

We use Shepp-Logan head model as the test image. 4 level 
pyramid structure is used. The original image is 114x114 
pixels, then we obtain pyramid images of 58X58, 30X30, 
16X16 pixels. The line integrals of object have been 
calculated according to the parallel ray geometry with 
16x16 digitization. Hence, a test image vector is 
x(256). 

In order to compare different algorithms for singlr cri- 
terion optimization and vector optimization fairly, we 
have used three measures of performance. These measures 
emphasize different aspects of the image quality, d is 
normalized mean root squared error measure, r is nor- 
malized mean absolute error measure, e is worst case 
normalized error. The results of computer simulation 
using VAX-1 1/730 is shown in Table 2. 
From Table 2 it is obvious that: 

( 1 ) Multicriterion method has higher image quality 
than all single criterion methods. 

(2) Multicriterion method takes less time to recon- 
struct an image. 

(3) Pyramid approach of image reconstruction is 
effective. 



531 



Table 1 
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Fig. 1. Pyramid Image Reconstruction. 
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Table 2 



ALGORITHMS 


d 


r 


e 


ITERATION 
NUMBER 
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TIME^ec 
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ruQon 
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5 
20 


252 
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INTRODUCTION 

In recent years, the engineering 
workstation has emerged as a separate 
category of computer system,- residing between 
higher-end personal computers and 
minicomputers or mainframes. The CPU and 
graphics capabilitities of these computers 
may provide the hardware bases for both 
surface and volume rendering approaches not 
only for engineering applications but also 
for use in the teaching and practice of 
medicine. 

Background 

A main distinguishing feature of the 
vo rkstation is the availabity of high 
performance graphics subsystems particularly 
optimized for 3D visualization. 
Additionally, these systems have a very large 
number of color memory planes and support 
displays having in excess of one million 
pixels. These graphics subsystems are 
connected to general purpose Unix-based host 
computers which in turn support the 
development and maintenance of large data 
sets associated with medical visualization. 
Within the workstation market there is range 
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of features and performance in the graphics 
systems; however, we will focus on the high- 
end graphics 3D workstation. The example 
system used in this article is the TurboSRX 
graphics system from Hewlett-Packard Company. 

The development of high-performance 
graphics systems has been significantly 
motivated by the mechanical engineering 
market. Large manufacturing concerns that 
use sophisticated design approaches to 
develop and prototype components rely on 3D 
workstations. The availability of realistic 
computer graphic representation of components 
is thought to reduce the need for extensive 
prototyping and accordingly accelerates the 
design process. 

In order to promote realism in design, 
surface or polygonal representations are used 
and rendered with specialized shading and 
lighting models. For the most efficient use 
of the engineers' time, interactivity is also 
crucial. Therefore, not only do the 
graphical representations have to be of the 
highest image quality, but also must be 
easily and rapidly rotated and translated. 
Specialized features including transparency, 
pan and zoom, and image blending enhance the 
users 1 ability to recognize complex 3D 
relationships. Understanding these complex 
spacial relationships are not only useful for 
the engineer working with complex mechanical 
objects, but also for the medical student and 
clinician in enhancing their appreciation of 
complex medical anatomy. As a result, the 
same graphical methods and features that are 
useful for the engineer may also be useful 
for members of the medical profession. 



535 



The type of solid model representation 
that is commonly associated with these design 
efforts is based on a surface model, one in 
which the hypothetical component is defined 
explicitly by a polygonal (triangular) mesh. 
Each triangle can be shaded, colored, and 
lighted. In the case of medical imaging, the 
particular anatomy is not a hypothetical 
structure, but is defined by a set of MRI or 
CT slices. These surface models can also be 
used in this environment, once the surface 
has been appropriately defined by 
thresholding or contouring. 

The entire data set can also be viewed 
using volumetric rendering approaches that 
may not require explicit contour-based 
segmentation. Various volume rendering 
approaches can be implemented using 
specialized and more general purpose 
computing solutions . 

Discussion 

If 3D graphics workstations from major 
vendors provide the necessary computing 
environment for the medical user, some 
distinct advantages result. For example, 
because of competition between hardware 
vendors, price-performance points may be 
superior as contrasted to that offered by 
smaller suppliers. Additionally, this same 
competitive situation may result in earlier 
implementation of state-of-the art chip 
technology in support of both graphics and 
general purpose computation. These advantage 
affect both image quality and realism as well 
as interactivity. 

Most of the emphasis in workstation 
design has focused on polygonal rendering 
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speed with associated features that promote 
image realism. The specialized geometry 
pipelines with their high-performance 
floating point transform engines and scan 
converters are now providing triangle 
rendering rates in excess of 100,000 per 
second. These capabilities support 
interactive surface polygonal anatomic 
representations. These graphical 
representations can be manipulated in real- 
time while not sacrificing assess to 
graphical features such as transparency and 
complex directional, positional, and ambient 
lighting models. The availability of the 
polygonal vertex data allows cutting routines 
that reference the original voxel data set as 
a basis for the rendering of the cut surface. 
These cutting capabilites, which are also 
interactive, expanded the utility of 3D 
visualization by allowing internal explcit 
and sharp surface views. (1) 

Volumetric visualization has provided 
extraordinary definition of medical volume 
data sets. Particularly useful has been the 
ability to represent objects that have no 
explicit boundary suitable for contouring. 
Multiple material models with translucency 
produce excellent images of such diffuse 
internal structures. Volume rendering can be 
performed on the same general purpose 
workstation as has been discussed above. One 
method relies on sampling a voxel subset that 
defines the material transition region., The 
voxel subset reduces the computational 
intensity to an extent that can be performed 
by the system CPU in a reasonable time. 
Another method which allows the use of the 
entire data set relies upon hardware assisted 
frame buffer compositing. (2) 



537 



Conclusions 

For the graphics workstation to provide 
the basis for a complete solution in the 
medical environment, both surface and volume 
rendering need to be addressed. These 
methods are not competing methods but each 
offers special and unique approaches to 3D 
visualization. Contemporary 3D graphics 
workstations may have the necessary hardware 
to support both complimentary imaging 
modalities, leaving it up to the software 
developer to take advantage of those 
capabilities . 
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1. Introduction 

Computed tomography is a well established imaging modality. Commercial 
CT scanners typically use analytic reconstruction algorithms which are based on the 
filtered backprojection technique. The digital filter can be viewed as the application 
of a window function to regulate the intrinsic ramp frequency response [1]. There 
is a general agreement that a window function tuned to the imaging task will 
give a better reconstruction man one selected arbitrarily and that smoothing out 
the intrinsic ramp filter response by an additional window function is in general 
beneficial [2,3,4,5], Assesment of algorithm performance arc, traditionally, based 
on subjective evaluation, typically by visual examination and comparison of a 
few images and their reconstructions. The number of objective evaluation studies 
performed on the role of the window functions are rather limited [4,5,6,7]. There 
is also considerable confusion over the question of whether the window function 
is essential. Some believe that filtering during reconstruction can be replaced 
with appropriate post-reconstruction filtering. Others believe, however, that due 
to finite sampling, this is in fact not feasible in actual CT. Because of the lack of 
satisfactory objective measures for algorithm evaluation, this disagreement has not 
been convincingly resolved. 

The Hotel ling trace criterion (HTC), a figure of merit of class separability [8], 
has recently emerged as a promising performance measure for evaluating medical 
imaging systems [9]. Based on the HTC approach, we have developed a procedure 
for objective evaluation of image reconstruction algorithms [10]. The procedure 
assumed a tumor detection task (this is equivalent to a binary classification task for 
discriminating the tumor-present class from the tumor-absent class) and the evalu- 
ation was based on statistical data sets of phantom images simulating the human 
thorax, with tumors randomly placed in predefined tumor sites. The same proce- 
dure was adopted in this study. A training set (with 252 images) and two testing 
sets (each with 126 images) and the necessary data were constructed to evaluate 
how the choice of the window function and the interpolation affect the performance 
of the filtered backprojection reconstruction algorithm. The algorithm performance 
was quantified by how well the optimal linear classifier could perform the tumor 
detection task, using optimal linear features derived from images reconstructed by 
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the algorithm. Selected post-reconstruction filterings were also performed on those 
reconstructions that were based on the ramp filter and were evaluated as above. 
This latter study was designed to answer the question of whether the window func- 
tion is essential in the filtered backprojection reconstruction algorithm. We report 
our experimental results and findings in this paper. 

2. Data and Methods 

The data sets used in the reconstruction and evaluation studies were based on a 
phantom image simulating the human thorax (Fig. 1), with an image size of S3xS3 
pixels. Bony structures including the sternum, vertebra and four ribs were simulated 
with ellipses and circles of appropriate size, all with a density value of 0.4. Twenty 
potential tumor sites, ten on the left side of the thorax and ten symmetrically on 
the right, were also defined. Five sites out of ten on one side were randomly 
selected and placed with a tumor. The five sites on the other side defined by the 
minor images of the unoccupied sites were also placed with a tumor. The tumor 
was assigned a density value of 0.205, versus a value of 0.2 for normal tissues. 
Altogether, 252 images, each with a unique combination of the ten occupied tumor 
sites, were generated. Two additional sample sets, each with 126 images randomly 
selected from the 252 image set, were also defined. Projection data of these image 
data with parallel geometry were generated using SNARK89 [11], with 180 views 
equally spaced over 180 degrees and a ray spacing of 1.0 pixel. Poisson noise were 
independently added to these projection data to generate three noise levels with 
averages of 1000, 2000 and 4000 photons per ray. 

The above procedure was repeated to generate a similar collection of projection 
data except with a finer ray spacing of 0.5 pixel. The three noise levels were 
adjusted to have, respectively, averages of 500, 1000 and 2000 photons per ray to 
compensate for the doubling of the number of rays in the new projection data. 

Based on the above projection data, images were reconstructed using the 
filtered backprojection (also referred to as convolution) algorithm with the choice 
of linear interpolation and one of the following filters: the ramp, Shepp and Logan, 
cosine (ramp with a cosine window) and Hamming (ramp with a Hamming window) 
[1,12,13,14]. Zero order (nearest neighbor) interpolation was also used for the ramp 
and the cosine filters. All of the above filters were defined with a cutoff frequency 
equal to the Nyquist frequency of sampling with ray spacing of 1.0 pixel. 

To test whether filtering by an additional window function during reconstruc- 
tion is the same as post-reconstruction filtering by the same window function, the 
following experiments were performed. The two dimensional discrete Fourier trans- 
form was applied to images reconstructed using the ramp filter. The Fourier co- 
efficients were filtered by the frequency response of the cosine window and the 
inverse Fourier transform was then applied to generate the post-reconstruction fil- 
tered images. Zero padding was used in the discrete Fourier transform to minimize 
inter-period interferences. These images were then compared with those recon- 
structed using the cosine filter. 
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soft tissue 0.2 
O unoccupied tumor site 0.2 
© occupied tumor site 0.205 
0 bony structures 0.4 



Figure 1 A thorax phantom with the non-overlapping sub-images illustrated. 
Each sub-image is a 44 pixel region centrally enclosing a potential tumor site. 

The above reconstructed images were evaluated using a tumor detection task 
and the performance of the optimal linear classifier was used as the performance 
index. To generate a two class data set for our binary classification task (discrim- 
inating the presence or the absence of a tumor), 20 non-overlapping subimages, 
each defined by a 44 pixel region around a potential tumor site (see Fig, 1), were 
extracted from each reconstructed image. The tumor-present (tumor-absent) class 
encompassed all those subimages where a tumor was present (absent). 

For each setting of imaging parameters (a given choice of reconstruction filter, 
interpolation, projection data sampling rate and noise level), reconstructed images 
of the 252 image set (which formed a two class data set with 1260 objects each) was 
used as the training set to determine the optimal linear feature operator. For binary 
classification, only one feature was needed [8]. This optimal feature operator, which 
we obtained using a multidimensional biased random search technique [15], thus 
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defined a single feature which maximized the class separability measure between 
the two object classes. This optimal class separability measure (the HTC) was 
adopted as a figure of merit for quantifying the performance of the reconstruction 
algorithm. Another figure of merit that we adopted was the area under the ROC 
curve; the ROC curve was determined by the maximum likelihood estimate [16], 
using output of the optimal linear feature operator as the decision variable. 

The optimal linear feature operators, one for each setting of imaging param- 
eters, were then applied correspondingly to the two testing sets to determine the 
two figures of merit for quantifying the performance of the given algorithm. The 
same evaluation procedure was applied to the post-reconstruction filtered images. 
To investigate whether significant errors were introduced in the post-reconstruction 
filtering process merely as a result of finite implementation, post-reconstruction 
filtered images and the pre-reconstniction filtered images based on some form of 
idealized projection data were compared. The projection data chosen for this study 
was the noiseless data with the finer ray spacing of 0.5 pixel. 

3. Experimental Results 

The algorithm performance given by the HTC measure for various filter 
functions and the two choices of interpolation are tabulated in Tables 1 and 2. 
Results in Table 1 are based on the projection data sets with ray spacing of 1.0 
pixel whereas those in Table 2 are with ray spacing of 0.5 pixel. All the three 
noise levels in each case are shown. In the tables, the first letter of the two-letter 
code denotes the filter function used with R for ramp, S for Shepp and Logan, C 
for cosine and H for Hamming. The second letter denotes the interpolation used 
with Z for zero order and L for linear. 



Table 1 The HTC performance measure for various choices of filter and 
interpolation using the projection data sets with ray spacing of 1.0 pixel. 



Noise Level 
(spacing=1.0) 


RZ 


RL 


SL 


CL 


HL 


1000 


Training 


0.101 


0.140 


0.141 


0.142 


0.143 


Testing 1 


0.106 


0.139 


0.141 


0.148 


0.150 


Testing 2 


0.103 


0.148 


0.150 


0.149 


0.148 


2000 


Training 


0.211 


0.285 


0.288 


0.298 


0.305 


Testing 1 


0.181 


0.249 


0.254 


0.263 


0.266 


Testing 2 


0.187 


0.267 


0.270 


0.272 


0.273 


4000 


Training 


0.419 


0.573 


0.582 


0.610 


0.619 


Testing 1 


0.366 


0.475 


0.486 


0.512 


0.524 


Testing 2 


0.366 


0.505 


0.515 


0.543 


0.547 
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Table 2 The HTC performance measure for various choices of filter and 
interpolation using the projection data sets with ray spacing of 0.5 pixel. 



Noise Level 
(spacing=0*5) 


RZ 


RL 


SL 


CL 


500 


Training 


0.139 


0.159 


0.160 


0.161 


Testing 1 


0.106 


0.121 


0.121 


0.130 


Testing 2 


0.096 


0.112 


0.113 


0.118 


1000 


Training 


0234 


0.282 


0 284 


0 286 


Testing 1 


0.232 


0277 


0278 


0.277 


Testing 2 


0.216 


0.263 


0.263 


0.265 


2000 


Training 


0.483 


0.597 


0.601 


0.608 


Testing 1 


0.475 


0.599 


0.601 


0.602 


Testing 2 


0.479 


0.602 


0.606 


0.605 



In Tables 3 and 4, both figures of merit, the HTC and the area under the 
ROC curve, are tabulated for the post-reconstruction filtered images based on the 
projection data sets of ray spacing of 1.0 pixel and 0.5 pixel respectively. A 
third letter C is added to denote the cosine window function used for the post- 
reconstruction filtering. The two performance measures for the ramp and cosine 
filters are also tabulated along side to ease comparison with those using post- 
reconstruction filtering. Only the two lower noise levels in each case are shown. 

In Table 5, we tabulated a few metrics derived from the following reconstructed 
images: ramp with zero order interpolation, the former plus post-reconstruction 
filtering with the cosine window, cosine with zero order interpolation, ramp with 
linear interpolation, the former plus post-reconstruction filtering with the cosine 
window and the cosine with linear interpolation. All these reconstructions are based 
on (Hie of the phantom and the noiseless projection data with the ray spacing of 0.S 
pixel. The metrics for the phantom itself are also tabulated. The metrics distance 
and relative error [17] are commonly referred to as the normalized root mean square 
distance and the mean absolute distance, both with respect to the phantom image. 
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Table 3 The HTC and A, performance measures for the ramp 
filter, cosine filter and the ramp with post-reconstruction using the 
cosine window, for the projection data with ray spacing of 1.0 pixel 



Noise Level 
(spacing=1.0) 


RZ 


CZ 


RZC 


RL 


CL 


RLC 


2000 


Training 




0.211 


0.270 


0.219 


0.285 


0.298 


0.289 


A z 


0.741 

±0.007 


0.769 

±0.007 


0.746 

±0.007 


0.775 

±0.006 


0.779 

±0.006 


0.777 

±0.006 


Testing 1 


rill* 


0.181 


0.240 


0.187 


0.249 


0.263 


0.252 


A z 


0.729 
±0.01 


0.755 
±0.009 


0.733 
±0.01 


0.763 

±0.009 


0.771 

±0.009 


0.765 

±0.009 


Testing 2 


H 1 V- 


0.187 


0.249 


0.197 


0.267 


0.272 


0.267 


Az 


0.730 
±0.01 


0.760 
±0.009 


0.736 
±0.01 


0.767 
±0.009 


0.770 
±0.009 


0.765 
±0.009 


4000 


Training 


HIV 


0.419 


0.550 


0.432 


0.573 


0.610 


0.587 


Az 


0.820 
±0.006 


0.853 
±0.005 


0.825 
±0.006 


0.858 
±0.005 


0.863 

±0.005 


0.861 

±0.005 


Testing 1 


HTC 


0.366 


0.449 


0.382 


0.475 


0.512 


0.489 


Az 


0.803 
±0.009 


0.829 
±0.008 


0.808 
±0.008 


0.835 
±0.008 


0.845 
±0.008 


0.837 
±0.008 


Testing 2 


HTC 


0.366 


0.497 


0.394 


0.505 


0.543 


0.522 


Az 


0.802 
±0.009 


0.841 

±0.008 


0.812 
±0.008 


0.843 
±0.008 


0.852 
±0.007 


0.848 
±0.008 
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Table 4 The HTC and Ax performance measures for the ramp 
filter, cosine filter and the ramp with post-reconstruction using the 
cosine window, for the projection data with ray spacing of 0.5 pixel. 



Noise Level 
(spacing=0.5) 


RZ 


CZ 


RZC 


RL 


CL 


RLC 






HTC 


0.234 


0574 


0.236 


0.282 


0.286 


0.287 




Training 


Az 


a nci 
0.753 

±0.007 


A T7A 

0.770 
±0.007 


0.754 
±0.007 


A HHA 

0.774 
±0.006 


A HHA 

0.774 
±0.006 


A *77C 

0.775 
±0.006 






HTC 


0.232 


0.270 


0.229 


0.277 


0.277 


0.276 


1000 


Testing 1 


Az 


0.753 
±0.01 


0.770 
±0.009 


A TM 

0.752 
±0.01 


0.773 
±0.009 


A TTA 

0.770 
±0.009 


0.772 
±0.009 






HTC 


0.216 


0.250 


0.209 


0.263 


0.265 


0.264 




Testing 2 


Az 


0.746 
±0.01 


0.763 
±0.009 


0.740 
±0.01 


0.766 
±0.009 


0.764 
±0.009 


0.768 
±0.009 






HTC 


0.483 


0.564 


0.487 


0.597 


0.608 


0.608 




Training 


A Z 


±0.006 


a oca 
0.856 

±0.005 


A Olfl 

0.839 
±0.006 


0.862 
±0.005 


A Q£.A 

0.864 
±0.005 


A OdZA 

0.864 
±0.005 






HTC 


A AK 

0.475 


A CCH 

0.557 


A A£.1 

0.462 


A CAA 

0.599 


0.602 


0.601 


2000 


Testing 1 


Az 


0.836 
±0.008 


0.853 
±0.007 


0.832 
±0.008 


0.863 
±0.007 


0.865 
±0.007 


0.864 
±0.007 






HTC 


0.479 


0.540 


0.459 


0.602 


0.605 


0.602 




Testing 2 


Az 


0.838 
±0.008 


0.851 

±0.008 


0.831 

±0.008 


0.866 
±0.007 


0.867 

±0.007 


0.866 

±0.007 



Table 5 Metrics for selected reconstructions of a single phantom 
using noiseless projection data with ray spacing of 0.5 pixel. 



Metric 


Phantom 


RZ 


CZ 


RZC 


RL 


CL. 


RLC 


average 


0.1155 


0.1155 


0.1155 


0.1155 


0.1155 


0.1155 


0.1155 


distance 


0.0 


0.0696 


0.0888 


0.0887 


0.0563 


0.0894 


0.0896 


relative 
error 


0.0 


0.0423 


0.0344 


0.0335 


0.0325 


0.0299 


0.0300 


variance 


0.0107 


0.0109 


0.0104 


0.0104 


0.0109 


0.0104 


0.0104 
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4. Discussion 

Some interesting results can be observed from Tables 1 and 2. For a given 
reconstruction filter, the algorithm performance given by the HTC measure more or 
less doubled as the photon count in the projection data doubled. This is consistent 
with the fact that the HTC measure is inversely proportional to the variance of the 
reconstructed pixel densities [9]. The algorithm performance is seen to increase 
with the smoothness of the reconstruction filter (filters in the tables were arranged 
with increasing smoothness from left to right). Improvements in lesion detectability 
with smoother filters have also been reported elsewhere [4,5]. For our data, a filter 
smoother than the Hamming filter did not seem to have as good a performance. 

A rather interesting finding is that the most dramatic improvement in algorithm 
performance occurred with the choice of the interpolation. Using the area under the 
ROC curve (A z ) and its error estimate (<r(A 2 )) given in Table 3, the use of linear 
interpolation with the ramp filter is seen to provide a very significant improvement 
(about 4.0 (t(A z )) in algorithm performance over that with zero order interpolation. 
As a comparison, when linear interpolation was used, the performance achieved 
with the rather smooth cosine filter is only one standard deviation better than that 
with the intrinsic ramp filter. For the results based on the finer sampled projection 
data (Table 4), using linear versus zero order interpolation with the ramp filter results 
in a lesser but still rather significant improvement (about 3.0 c(A z )), whereas the 
choice of filters did not seem to matter much when linear interpolation was used. 
Evaluation based on the HTC measure, though we do not have an error estimate, 
also leads to similar conclusion. 

Even though the results above point to the choice of a smooth filter and linear 
interpolation and the fact that a proper choice results in a significant improvement 
in algorithm performance, we must emphasize that the relative performance of 
various reconstruction filters also depend on other imaging parameters such as the 
sampling rate and noise level in the projection data, the objects to be reconstructed, 
the imaging task to be performed, etc. 

The results given in Tables 3 and 4 served to answer the question of whether 
pre-reconstruction filtering can be equivalently performed with corresponding post- 
reconstruction filtering. The cosine filter with both choices of interpolation was 
chosen for this study. For the results based on the projection data sampled 
at 1.0 pixel spacing (Table 3), the cosine filter is seen to provide significant 
improvement over the ramp filter for both zero order and linear interpolation 
(respectively by 3.0 <r(A z ) and 1.0 <r(A z )). Post-reconstruction filtering with the 
cosine window on images reconstructed by the ramp filter is seen to provide slight 
improvement on both performance measures (by about 0.S standard deviation for the 
A z measure) for both choices of interpolation. For the case of linear interpolation, 
since the performance of the cosine filter is only one standard deviation better 
than that of the ramp, we can not say with statistical significance that there is 
a genuine difference between pre-reconstruction filtering and corresponding post- 
reconstruction filtering. For the case with zero order interpolation, however, one 
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can see that pie-reconstruction filtering using the cosine filter is significantly better 
than the post-reconstruction filtering using the cosine window. 

For the results based on the finer sampled projection data (Table 4), there is 
no significant difference in performance between the ramp filter, the ramp filter 
with post-reconstruction filtering using the cosine window and the cosine filter, 
for the case of linear interpolation. For zero order interpolation, the cosine filter 
still performed significantly better than the ramp filter (by about 2.0 <r(A z )). Post- 
reconstruction filtering on the reconstructions by the ramp filter seems to result 
in a slightly poorer, though not significant, performance. Once again, the pre- 
reconslruction filtering using the cosine filter is significantly better than the post- 
reconstruction filtering using the cosine window. 

Aliasing artifacts [18], to which sharp reconstruction filters are more prone, 
are errors that we believe could not be effectively removed by post-reconstruction 
filtering. This is believed to contribute, at least in part, to the observed poorer 
performance associated with the post-reconstruction filtered images. The above 
arguments would lead to an expectation that post-reconstruction filtering for the 
finer sampled projection data with 0.5 pixel spacing (with less aliasing) would 
result in more improvement than for the data with 1.0 pixel spacing, given all 
other imaging parameters the same. The above observed results seemed to fail 
our expectation. This latter conclusion is not necessarily true due to a design flaw 
in the experiments. Even though the experiments were set up with the aim that 
the projection data sampling rate was the only variable, a closer look revealed a 
subtle difference in the overall frequency response of the filter used for the two data 
sets. This arose because the frequency response resulted from interpolation actually 
depended on the Nyquist sampling frequency. The response associated with zero 
order interpolation for the finer sampled data as a result was much sharper. This 
could have led to what we have observed. 

A natural question that arises is whether the observed significant difference 
between pre-reconstruction filtering and corresponding post-reconstruction filtering 
is not merely due to errors introduced by the finite implementation of the post- 
reconstruction filtering process. The results given in Table 5 addressed this issue. 
The table listed a number of metrics derived from relevant reconstructed images 
(and the phantom) of a single phantom based on noiseless projection data with the 
finer spacing of 0.S pixel (a form of idealized data). For both cases of interpolation, 
one can easily see quite a difference in the metrics between that of the ramp and the 
cosine filter. For the case of linear interpolation, the post-reconstruction filtering 
with the cosine window resulted in metrics more or less identical to those using the 
cosine reconstruction filter. That is to say, the finite implementation of the post- 
reconstruction filtering process did not result in a significant error. For the case of 
zero order interpolation, the same is true for most metrics except for the relative 
error measure which amounts to say, for the given data, that the post-reconstruction 
filtered image still differs from that reconstructed from the cosine filter. This could 
well happen given the relative sharp response of the ramp filter with zero order 
interpolation. 
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In summary, we have shown that based on a tumor detection task and an 
objective evaluation using the optimal linear classifier, window functions are both 
necessary and essential for the filtered backprojection reconstruction algorithm. A 
proper choice of the reconstruction filter can result in a significant improvement in 
algorithm performance and that filtering effected by the window function associated 
with the filter can not always be equivalently replaced with post-reconstruction 
filtering using the same window function. 
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A Numerical Two- Alternative -Forced -Choice (2AFC) 
Evaluation of Imaging Methods 
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1. Introduction. The two-alternative-forced-choice (2AFQ approach 
to evaluating the comparative quality of two or more imaging methods 
consists of presenting observers with pairs of images, exactly one of which 
is known to have a signal (tumor, abnormality, etc.) in it The observer 
selects one as signal containing. The respective percentages of correct 
hits are used as figures-of-merit (FOM's) of the imaging systems. Under 
appropriate conditions, this FOM approximates the area under the receiver- 
operating-characteristic (ROC) graph [1]. 

To avoid the time and expense of using human observers in a 2AFC 
study, one can perform a numerical version of the same. Here one 
associates a computable abnormality index with the images and selects 
the image with the higher abnormality index [2]. 

We designed such a numerical 2AFC experiment to evaluate the 
comparative performance of image reconstruction methods for detecting 
increased uptake in Positron Emission Tomography (PET). Mathemati- 
cally described phantoms approximating structures in the brain at various 
transaxial levels were used. The structures are symmetrical with respect 
to the midline. For each pair of symmetrical structures in the phantoms, 
one of the pair was randomly selected to have a higher uptake than the 
other. A total of 104 phantoms were generated containing 856 pairs of 
structures. The data collection (including noise) of a PET scanner was 
simulated and each of the phantoms was reconstructed by various image 
reconstruction methods, which were then evaluated using numerical 2AFC, 
by selecting that structure of a pair which has the higher total activity in 
it in the reconstructed image. 

We report on our experiments comparing the classical convolution (or 
filtered backprojection) method with expectation maximization (EM) algo- 
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rithms both for maximization of likelihood (ML) and for maximum a 
posteriori probability (MAP) estimation. In addition to reporting on the 
results of the 2AFC evaluation (a decision task) we also report on the 
results of a distance-measure evaluation (an estimation task). 

2. Data Generation and Evaluation Methodologies. In order to 
make our experiments realistic from the point of view of clinical imaging 
tasks, we used a computerized overlay atlas based on average anatomy 
[3]. This atlas consists of 26 overlays each corresponding to a transaxial 
slice of the brain. Neuroanatomic structures are represented by ellipses and 
rectangles at the appropriate locations; they are symmetrical with respect to 
the midline. We made use of this symmetry to set up our 2 AFC experiment 
as follows. 

We created mathematical phantoms in which (in arbitrary units) the 
activity in the brain background is 1.00 and in the neuroanatomic structures 
it is 1.9S or 2.00. For each pair of symmetric structures a random number 
generator chooses exacdy one of the pair to have activity 2.00. Thus the 
task is to decide, based on the reconstructions, which one of the pair of 
structures has the higher activity. The top left image in Fig. 1 shows one 
such phantom. 

We generated images and projection data of these phantoms using 
SNARK89 [4]. In generating the projection data we assumed a ring of 300 
detectors with each detector in coincidence with 101 detectors opposite it 
(In SNARK89 terminology, we have a divergent ray geometry with 300 
views and 101 rays per view.) Poisson noise was introduced into the 
measurements on the basis of a total 3X10 6 photons collected per slice 
(averaging approximately 100 photons per SNARK89 ray). The resulting 
ray sums were further deteriorated by a 10% multiplicative Gaussian noise, 
to take care of other sources of error besides photon statistics. 

The set of 26 overlays was reused four times; the sides of elevated 
activity and the noise in the data were chosen independently for the four 
data sets. Thus a total of 104 phantoms (and their projection data) were 
generated; they contain a total of 856 pairs of symmetric structures. 

The FOM of a reconstruction method provided by 2AFC for such a 
data set is defined as follows. For each structure we define the abnormality 
index in the reconstruction to be the sum of reconstructed pixel values for 
those pixels whose centers are within that structure in the phantom. A pair 
of symmetric structures provides us with a hit, if the abnormality index 
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in the reconstruction is higher for that structure in the pair for which the 
activity is higher in the phantom. The FOM is the ratio of hits to the total 
number of pairs in the data set. 

Since most image reconstruction methods have some free parameters 
in them, it is appropriate prior to evaluation to train the method for a 
particular task [2]. For this reason, we designated one of the four data sets 
(each consisting of 26 phantoms and their projection data) to be a training 
set and the other three to be testing sets. The free parameters were chosen 
so that they give the highest possible FOM on the training set, and then the 
methods were evaluated with these parameters fixed on the (statistically 
independent) testing sets. 
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Figure 1. One of ihc phantoms and its reconstructions. 
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In addition, we decided to evaluate the performance of the recon- 
struction methods from another point of view: namely, on the basis of 
their ability to estimate the total uptake in the neuroanatomic structures. 
For this we used the so called clipped normalized distance provided by 
SNARK89 [4]. Briefly, for this measure both the phantom and recon- 
struction are clipped, by setting pixel values which are less than (in our 
case) 1.65 to 1.65 and pixel values which are more than (in our case) 2.05 
to 2.05. After this a normalized root mean square distance between the 
clipped reconstruction and clipped phantom is calculated. The clipping 
insures that only the accuracy within neuroanatomic structures contributes 
to the distance measure. 

3. Experimental Results. The methods that we compared were the 
convolution (also known as filtered backprojection) method (see, e.g., [5]) 
and two EM-type algorithms, one for maximizing likelihood (ML) and the 
other for maximum a posteriori probability (MAP) estimation (see, e.g., 
[6]). For the latter we chose the prior described in [7], which penalizes 
for differences between a pixel and the average of its eight neighbors. We 
used the computer implementation described in [7]; in the case of ML this 
results in a pure EM algorithm without any ''acceleration'* or other tricks. 
Because of this, there were no free parameters to optimize for the ML case. 

For the convolution method we had to select a filter. Using the training 
set, we found that a very smoothing filter is the most appropriate for the 
decision task; it is one which sets all frequencies to be zero beyond eight- 
tenths of the Nyquist rate (in SNARK89 terms: cutoff=0.8) and otherwise 
has the shape of the generalized Hamming filter with a=0.5 [5]. This is 
quite different from what has been found most useful for x-ray CT, which 
would typically be a generalized Hamming filter with a=0.8 and cutoff=1.0 
[4,5]. Reconstructions using these two filters are shown in the middle and 
on the right, respectively, in the top row of Fig. 1. 

For the MAP estimation we had to select the weight 7 to be given to 
the smoothing prior [6,7]. Using the training set we found 7=10 to be the 
most appropriate for the decision task. The middle and the bottom rows 
of Fig. 1 show reconstructions by the EM algorithms, for ML and MAP, 
respectively, after 15, 45 and 145 iterations. 

In Table I we report on the hit ratios for the three testing sets for the 
convolution method with the two different filters and for the second and 
the twentieth iterations of the two EM algorithms. In Fig. 2 we report 
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Figure 2. The averages over Ihc three testing sets of the 
hit ratios (top) and clipped normalized distances (bottom). 
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on the averages (over the three testing sets) of the hit ratios and of the 
clipped normalized distances. 



Testing Set Number 


1 


2 


3 


convolution (optimal) 


.762 


.822 


.832 


convolution (a=.8, cutoff=l) 


.715 


.785 


.790 


both EM's (iteration 2) 


.822 


.869 


.832 


EM (ML, iteration 20) 


.766 


.832 


.799 


EM (MAP, iteration 20) 


.771 


.832 


.804 



Table 1. Hit ratios for the testing sets. 



4. Discussion. Our results can be summarized as follows. 

1. Early iterates (up to about fifteen) of the two EM algorithms perform 
very similarly for both tasks. With increasing number of iterations 
the performance of ML optimization deteriorates relative to MAP 
estimation. 

2. For the 2AFC (decision) task, the second iterate of the EM algorithms 
was the best and it outperformed the (task-optimized) convolution 
method. However, the performance of both EM algorithms deterio- 
rates with increasing number of iterations and gets to be worse than the 
(task-optimized) convolution method after the twenty-fifth iteration. 

3. For the estimation task, the two EM algorithms perform quite dif- 
ferently. For ML, the performance improves until the twenty-eighth 
iteration, after which it gets worse, never reaching the performance of 
the convolution method. For MAP, the performance improvement is 
monotonic, with performance becoming better than that of the convo- 
lution method after the fourtieth interation. 

Note that the performance of the methods is very task-dependent. The 
second iteration of EM is better than convolution for the decision task, but 
is worse than convolution for the estimation task. The reverse is true for the 
fortieth iteration of the EM algorithm for MAP compared with convolution. 
Hence, while numerical 2AFC is a perfectly reasonable approach if we are 
evaluating the comparative performance of two imaging methods for a 
decision task, one has to be careful not to jump to conclusions regarding 
the relative performance of the two methods for another task, especially 
for an estimation task. 
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For reasons of space we do not provide here a discussion of the 
statistical significance of our results; we refer the reader to [8]. 
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1 . Purpose 



Modern surgery requires exact planning. The radiologist not only has to 
state the diagnosis, but also to provide a maximum of spatial information for 
the surgeon. In order to acquire an optimal spatial perception, a variety of 
mapping modalities is at hand, including US, CT, MRI, and DSA. 3D 
information is contained in any set of tomographic slices. All the information 
is primarily gathered by the radiologist and, mentally built into a spatial 
image. Nevertheless it is difficult to explain spatial relation to a large forum 
(conferences) on the basis of cut - modality images. As every listener has to 
build up the image in his mind from the information of transversal cut 
modalities and additive modalities (DSA), the results often vary individually. 
The optimal way to present the spatial information would be to present a 
model of the relevant anatomical objects (organs, lesions, vessels, bones 
a.s.o.). As the construction of models is very time - consuming, the 3D 
display on monitors with shading characteristics is applied. This system is 
well established in bone surgery (spine, pelvis). It has, up to now, seldom 
been applied to the abdominal structures, which have a no less complicated 
spatial relation. Choosing the liver with focal lesion as an example, we want 
to show a few of the problems arising from 3D reconstruction of abdominal 
structures. 
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2. Method and Materia! 

Prior to surgery of primary or secondary focal liver lesions we routinely 
perform a liver CT - scan (Picker 1200 Expert) under transarterial 
portography (CTAP) conditions. Contrast media is selectively injected into 
the SMA. This study modification has proved to optimize lesion detectability 
by maximizing lesion / parenchyma contrast difference. At the same time, 
the portal venous system is maximally enhanced. 




Fig. 1. Tiancparcncy 3D image of the liver. The liver is translucent so that the portal 
branches and the focal lesion can be seen within the liver 

Analysing 12 of these CT scans, we reconstructed 3D images of the liver. In 
order to reconstruct the 3D appearance of an object of interest it is 
necessary to determine the pixels of the object in the slices. There are three 
possible ways to collect the pixels. Defining a range of HU will define the 
pixels not only of the chosen organ but also of several other areas of the 
slices. To avoid marking of unwanted pixels, the autocontour procedure is 
administered. With this procedure, a certain threshold, depending on the 
densities of the object, is chosen. Setting the cursor in the CT-slice and 
starting the procedure, the computer seeks in a straight line to the left of the 
cursor until she finds a pixel with a HU of the threshold value or higher. In a 
clockwise direction, the computer constructs a border between pixels with 
HU higher and HU lower than the threshold. By this means, even an 
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inhomogenous structure can be collected as long as a border can be 
defined. In some cases it is necessary to edit manually, i.e. to "draw" a part 
of the contour by hand. Once the border surrounds the chosen structures, 
all pixels inside are "collected" for reconstruction. 

This procedure is repeated in all slices and for all objects, where objects 
embedded inside another object have to be defined first, otherwise they 
would be added to the surrounding object (i.e. liver metastasis). 
We reconstruct three objects: the portal system, the tumors, and the liver 
parenchyma. These objects are then displayed simultaneously as a so 
called "transparency", for some objects are transparent in order to visualise 
the objects within (fig. 1). 




Fig. 2. The region of the liver is reconstructed as a block. The CT values are added to 
the surface. The liver is seen from anterior and left, from inferior the kidney and the 
spine can be seen clearly. 

A second method to achieve a 3D image, the so called block display, is to 
collect a block of data. This means to collect the data without special 
marking of objects. The whole block is reconstructed and the HU are shown 
on the surface of the block (fig. 2). The block can be cut in any plane, 
rotated and combined with the reconstruction of single objects (fig. 3). 
To show the object spinning or tumbling on the monitor is, in a certain way, 
still another 3D modality. This "3D - movie" can be performed with a 
transparency as well as with block data. 
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The reconstructed images and the 3D - movies are evaluated in regard of 
best spatial information, in relation to transversal CT scans and DSA. The 
evaluation is performed by 1 radiologist, 1 radiology resident, 3 surgeons, 2 
surgery residents and 2 medical students. Each study is graded from 1 (very 
good spatial information) to 6 (no spatial information). The surgeons were 
asked additionally to value the 3D display, if it was nessecary for the 
operation (grade 1) or did not help at all in planning and performing the 
operation (grade 6). After figuring the average grade for each modality for 
every test reader the grades were normated for comparison. 




Fig. 3. Transparency display and block display are combined. The right lobe of the liver 
is cut off, the left lobe is displayed as block with CT values added to the surface. Tumor 
and portal vein with branches arc added as distinct objects. 



3. Results 




To define an object of interest, manual contouring does not create 
results. Autocontouring does not determine the exact 
; in most cases, but is reproduceable. Autocontouring is sufficient to 
determine the boundaries of most focal liver lesions and of the portal system 
when performing CTPA. A combination of manual editing and 
autocontouring shows best results to determine the liver boundaries. None 
of our studies could be reconstructed without manual editing. 
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The average spatial information value of the different modalities as evaluated 
by the test readers is shown in the following table (grades already 
normated): 



Test reader 


DSA 


CT 


3D Image 


3D Movie 


3D Vali 


Surgeon 1 


4.8 


1.9 


2.7 


2.5 




ouryeun & 


A A 


o.u 


CO 




3 4 


Surgeon 3 


5.2 


3.0 


2.3 


2.2 


4.4 


Surg. Res. 1 


4.3 


2.7 


2.8 


2.3 


1.5 


Surg. Res. 2 


4.7 


3.1 


2.6 


2.4 


2.3 


Radiologist 


5.0 


2.8 


2.4 


1.9 




Rad. Res. 


4.2 


2.3 


3.0 


2.7 




Med. Stud 1 


4.6 


2.9 


2.2 


2.2 




Med. Stud. 2 


4.5 


2.4 


2.8 


2.4 





4. Discussion 



The 3D reconstruction of bone is by now a well established part of 
preoperative planning in the field of the spine, pelvis, and cranio - facial 
disorders. Because of the high contrast differences it is rather easy to 
determine the object margins and in most cases this can be peformed 
automatically with a high degree of reproduceability. 
The average HU of the anatomical objects in the abdomen vary significantly, 
but in most instances have a wide range and because of this fact overlap. 
Just choosing certain density levels and/or ranges is not sufficient to 
determine the anatomical structures. Autocontour procedures do help to 
increase the reproduceability, but in all our cases, manual editing was 
essential. Statistical boundary analysis and object recognition should be 
introduced for totally automated, reproducable contour finding. 
As the clinical diagnosis is established by other modalities, the 3D image 
does not need to be highly reproduceable as long as only spatial relations 
have to be shown. Actually the radiologist, editing the slices for 
reconstruction, already has to know what the result has to look like. 
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A basic problem in 3D development is the varying acceptance by the 
clinicans. Most surgeons, after a moment of enthusiasm, continue preferring 
the cut slice display they are used to. If the clinican is not that much used to 
reading CT's, as residents are, 3D is valued higher. Obviously there is no 
need for sophisticated imaging modalities if one modality can show that the 
patient is inoperable (multiple metastases in both liver lobes). With single 
lesions, or single lobe lesions, the advanced surgical possibilities call for 
more differentiated imaging and presentation. 

The clinicans still are waiting for the most convincing 3D - display. This 
expectation is to stimulate the further technical development and research. 



5. Conclusion 

The ideal imaging system displays anatomical objects as a whole. The user 
then may start peeling off, cutting away, radiographing parts of it and thus 
may almost realistically simulate the surgical operation. This would mean to 
have exact information of all objects with highest spatial resolution and to 
perform in real time. The system of displaying block data does have a 
potential to achieve this as soon as faster computers are readily available. 
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Transversal CT slices and 2D reconstructions 
do not always provide the optimal spatial in- 
formation of the complex f anatomical structures 
of the visceral cranium. The 3D CT introduced 
in recent years has gained increased popula- 
rity. The goal of this study was to evaluate 
the feasibility of the 3D CT in orofacial sur- 
gery. 



Patients: 

Threedimensional image reconstructions were 
generated in 40 patients with orofacial tumors 
and 12 patients with fractures (36 males ,16 
females, mean age 54 +- 9 years). 

Methods : 

Each study was performed with the Soma torn 
Plus, Siemens (continuous rotation of tube and 
detector) . Phantomstudies with 2 mm thick and 1 
mm overlapping slices were superior to 1 and 5 
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mm scans in threedimensional rende- 
ring. Therefore all scans were made with these 
parameters and 0,7 s scanning time (60 mas , 120 
kv).The time for scanning and threedimensional 
surface reconstruction lasted from 30-45 minu- 
tes. 

The 3D surface display offers a spatial repre- 
sentation through a shaded reproduction of the 
object .Pixels nearer the viewer are imaged 
brighter , while structures further away are 
imaged darker. Free selectable outlines permit 
the projection of any internal view. Image re- 
constructions of any projection require a mat- 
ter of seconds. 

Results : 

The image quality of the osseous structures 
was excellent without any artefacts in 24 
cases ( tab . 1 , f ig . 1 ) . The complex destructions 
or fractures were clearer and easier for the 
oral surgeon to understand than with transver- 
sal slices. A slight reduction of image quality 
was due to motion or metal artefacts in fur- 
ther 25 patients (fig. 2) .Sufficient 
threedimensional images could not be generated 
for three uncooperative patients. 
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Table 1: 



Image Quality of 3D CT: 



very good: 
good: 



poor: 



24 patients 

25 patients 
3 patients 



Discussion: 

In concordance with Vannier ,Grodd and Totty 
(1,2,3) our experience has shown, that 3D CT 
leads to an improvement of the delineation of 
complex osseous destructions .Additional spa- 
tial information can be achieved with this me- 
thod. But better diagnostic information of the 
visceral cranium cannot be obtained with the 
3D CT, because the image information of trans- 
verse CT slices provide the basis for this re- 
construction. Discrete bony alterations can 
only be demonstrated with certainty in trans- 
versal CT scans ( 1 ) . 

Thin overlapping slices, short scanning times 
and reduced mas products provide 3D CT images 
of optimum quality. 
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Fig.l: 

Cheek-bone and temporal bone fracture. 

Very good quality of the reconstructed image. 




Fig. 2: 

Fracture of the lower jaw. Slight reduced image 
quality because of metal artefacts. 
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1. INTRODUCTION 

The need of estimating relative displacements between 
the images taken at different times and conditions has 
arisen in a variety of engineering problems such as 
medical imaging, remote sensing and machine vision. A 
typical example is digital subtraction angiography. In 
DSA the vascular information are obtained by subtrac- 
tion of the mask and the live image. In principle, the 
difference image contains only the projection of blood 
vessels. However, visibility of the vessels is hampered 
by the presence of patient motion. Hence, for any tem- 
poral subtraction techniques of DSA, estimating the dis- 
placements and reregistrating the original images are 
necessary. 

The conventional methods of displacement estimation are 
based upon matching the gray level of the two images 
with different relative displacement, and the position 
with maximum similarity was selected as the optimum 
estimation. But in DSA due to the appearence of contrast 
agent in the live image, matching of gray level does not 
gave optimal estimation of displacement. Therefore opti- 
mal estimation of displacement must consider the gray 
level change due to contrast agent and other factors. 

Some works considered the gray level change caused by 
contrast agent through using a global gray level com- 
pensation. But these works did not consider the time 
and spatial varying property of the gray level change, 
therefore the results were unsatisfactory. 

2. GRAY LEVEL CHANGE MODEL 
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In order to consider the local distribution of the vessel 
system (filled with contrast agent). It is logical that 
a performance criterion should be some function of gray 
level change or unknown model parameters which stand for 
main features of such a change. To characterize the mean 
spatial-varying features of the vascular system fj, We 

divide the image plane into ( J-,+1 ) (K-,+1 ) subregions 

For each subregion search of optimum for some 

criterion R not only in two-dimensional space (A* .Ay) , 
but also the gray level change, i.e. searching is in a 
space of higher order (A*.Ay.f l0 ) * where f I0 is a model 

parameter reflecting gray level change. 

Thus let fj be the vascular system to be detected, then 
it can be approximated by f^: 

fii(x*.t)«S Efio(Ji.M) r i (1) 

li«o M»o 

where rj is a two-dimensional function defined over 
| bj 1 k 1 1 > fj<| is an estimation of fj. 

In the simplest case rj is a zero order function as: 

W ' y} lQ elsewhere ^ 

where the size of bj^ is e Q uaJ t0 h x h y # 

In the case vascular system fj is approximated by 
(J 1 +1)(K 1 +1) number of ^ io<3i »kl ) is the gray level 

change of subregion (j^ki). fyi can be considered as 
a low spatial frequency approximation of f j. 

3. ESTIMATION PERFORMANCE CRITERION 

The performance criterion used in our method is direct- 
ly dependent upon the difference of live image f c and 

mask image f m , so referred to as the minimum artifact 
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criterion. 

Considering the correction of gray level change, then 
error term of f and f m can be modified as 

e Jk (Ax.Ay.f n ) -f c Cx^)-f m Oc+Ax t y+Ay)-f ll (J,J{i) (3) 

Use some kind of error function of e^, 

ejk(Ax.Ay.f |1 ) = £(e Jk (Ax.Ay.f l1 )) (4) 

then 

RCAx.Ay^^kJJ-SS^jkC^JkCAx.Ay^^j.k)) (5) 

* k 7 

The error functions commonly used such as jej k J , (ej k ) 

have some drawback: terms of larger errors have more 
strong influence on criterion R than other smaller error 
terms. It would results false estimate of matching as 
strong interference appears, which is just the case of 
contrast agent in the vessel system. One of the error 
functions we used is as follow: 

5,k(4X,iy - , ") = A 2+ |e, 1 ,(A A x 1 .Ay.f l1 )| (6) 

which makes each term of R have more equal effect on R. 
4. PYRAMID APPROACH 

Increasing the order of searching space inevitably in- 
creases the time needed to estimate the optimal solu- 
tion, and will made the method unpractical. 

To solve this problem, we use an algorithm based on 
Gaussionpyramid image data. The search procedure pro- 
ceeds from the top to the bottom of the pyramids. The 
information obtained at the lower resolution level are 
used to guide and control the starting search conditions 
of vector z at the next higher resolution level of the 
pyramids. The flow chart of the algorithm is shown in 
Fig. 1. Where U 0 ,y 0 ) denotes the coordinates of the 
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window, used as the reference template, in f m (or f c ); 
(Uq,v q ) is the central coordinates of search area in f 
(or f m ); (u.v) is the estimated corespondence point of 
(x 0 ,y 0 ); (Ax,Ay) = (u,v) -(x 0 ,y 0 ); x p , y r is the space 
search range; I r is the gray level search range; I Q is 
the initial value of f I0 . 

5. EXPERIMENTS AND RESULTS 

The algorithms were tested on several pairs of X-ray 
images. 

Original information on X-ray films were converted into 
512 2 digital matrix denoted f°, from which a Gaussian 
pyramid image sequence: f 1 , f 2 , f 3 , f 4 , f 5 was generated, 
with the sizes: 256 2 , 128 2 , 64 2 , 32 2 , 16 2 , respectively. 

The matching computation were carried out only on f^-f^. 
The size of image windows used in matching be M?, L=1, 
2,3,4,5, then we took M 2 =64 2 , m|=64 2 , M 2 =32 2 , m|=16 2 , 
M 2 =8 2 . 

Started from f 5 , 16 x 16 pixels, 8x8 window, 4 subregions. 
Estimated gray level change values of the four subregions 
are: 

1 



The difference image at level 5 is: 
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The average gray values of the four subregions of the 
difference image at level 5 is: 

1.3 2.0 

Compare with the result of gray level change estimation, 
the accuracy is satisfactory. 

According to the R at different matching position, the 
estimated displacement at level 5 is 

Ax=3, Ay=3 

We use quardratic 2-D interpolation to modify the esti- 
mated displacement to an accuracy of 1/10 spatial reso- 
lution of the image and found the modified value at level 
5 is 

Ax^=--0.1,Ay # =~0.5 

These values are used to control the second iteration 
(level 4). The process of iteration is repeated until 
we find the displacement estimation at level 1. 

6. CONCLUSION 

(1) Estimation of local gray level change by divided 
the image into subregions is effective to treat the in- 
fluence of contrast agent in the vessel system. 

(2) Pyramid approach of multilevel estimation can 
accelerate the searching procedure considerably. 

(3) A hyperbola error function is better than |e| or 
e^ function in this case. 
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Fig. 1. Flow Chart of Pyramid Approach. 
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ABSTRACT 

We describe an algorithm that reconstructs the cross sections of the 
lumens of coronary arteries from two mutually orthogonal x-ray projec- 
tions. In other techniques of reconstructing these cross sections from 
biplane images it is assumed that the cross sections are elliptical. Al- 
though this assumption is valid in healthy portions of the artery, stenotic 
portions can have crescent or star shapes as well as elliptical. Our al- 
gorithm accommodates the possibility of such shapes. Our algorithm 
represents each biplane projection of a transverse slice of the arterial lu- 
men as a binary- valued image. The single- coordinate moments of these 
two binary- valued images are matched to those of an initial binary- valued 
image of the slice. Since the cross-coordinate moments of the slice can- 
not be matched to those of the biplane projections, we represent the slice 
as a sum of weighted Legendre polynomials. The weights are estimated 
by an iterative procedure involving singular value decomposition. Pre- 
liminary tests of our algorithm on synthetic data as well as on biplane 
images of a segment of a barium enhanced in vitro heart are reported. 

I. INTRODUCTION 

Knowledge of the shapes and sizes of the lumens of coronary arter- 
ies enables the estimation of several important clinical parameters [1]. 
In several techniques for 3-D reconstruction of coronary arteries it is 
assumed that the cross sectional shapes of the coronary arteries are el- 
liptic [2],[3],[4],[5]. Although this assumption is valid when the arteries 
are healthy, it may not be so for stenoses. Stenoses may have crescent or 
star-like shapes as well as elliptic [12]. In this paper, we suggest a means 
of reconstructing such a shape from two orthogonal x-ray projections of 
the arteries. We describe the underlying mathematical theory and report 
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the results of preliminary tests on synthetic data, and a segment of an 
in vitro heart. 

As in earlier work on this subject we view the three-dimensional re- 
construction of a coronary artery as the reconstruction of a stack of slices. 
In each slice, the cross section of the vessel is given by the density pro- 
file along each horizontal scan line. Assuming that the contrast medium 
within the vessel is distributed homogeneously, we represent each cross 
section as a binary-valued image, where pixels have the value of 1 if 
they belong to the vessel and 0 otherwise. Thus the problem of finding 
the cross-sectional area of the vessel in each scan line from the two or- 
thogonal projections is transformed into the problem of reconstructing a 
binary-valued image from its row and column sums. We represent such 
an image by a matrix whose elements are restricted to O's and l's. We 
refer to this as a binary matrix. 

The problem of reconstructing a binary matrix from its row and col- 
umn stuns is indeterminate, i.e there may be no, one or several solutions. 
Consider a binary matrix I with U rows and V columns. Let R(i) repre- 
sent the sum of row i, and let C(j) represent the sum of column j. Then 
we have: 

JZ(i) = £j(i,j) fori = h...,U 

j 

and 

= for j = h^V 

i 

When we try to solve for the value of each pixel within the matrix 
I, we usually encounter more than one solution since we have U times V 
variables and only U+V equations. 

II. PREVIOUS WORK 

In [6], Chang and Chow described a means of reconstructing a binary 
image from two orthogonal projections, based on the assumption that 
the object is convex symmetric. Later in [7], Chang and Wang showed 
that, for a convex symmetric binary object, if one contour position is 
known, the shape can be uniquely reconstructed from two orthogonal 
projections. 

Onnasch et aU [8] described a technique for reconstructing the left and 
right coronary ventricles from biplane angiograms, using the reconstruc- 
tion of an adjacent slice and of the same slice in an other phase of the 
cardiac cycle. The values of the pixels within the binary matrix were 
determined from an array of probabilities. Slump and Gerbrands [9] 
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described a method of finding a binary matrix that satisfy the two pro- 
jections with maximum resemblance to adjacent slices ( temporally or 
spatially ) based on a cost function. This method is called the minimum 
cost capacitated network flow algorithm. 

All of the above techniques were devised for reconstructing coronary 
ventricles* Reiber et al[l] adapted Onnasch's and Slump's algorithms 
into reconstructing coronary artery cross sections. Good results were 
obtained. A problem with this approach is that sometimes the a priori 
information does not exist, as in the case of the first slice. This can be 
circumvented by having the user select the slice where he/she is quite 
sure that the shape is elliptic. However, as stated in [1], in a stenotic 
segment the two neighboring slices of a particular slice might be very 
different. Our approach, described below, overcomes this difficulty. 

III. MATHEMATICS OF RECONSTRUCTION 

In this section, we present the mathematical theory supporting our 
approach to reconstructing a binary matrix from its projections. If the 
domain of F(x,y) is continuous the two-dimensional moment f pq is de- 
fined as 

fpg = J J x p y q F(x,y)dxdy (1) 
If the domain of F(x^ y) is discrete 

/« = EEw.») (2) 

x y 

If the function F(x^y) is piecewise continuous, bounded and has 
nonzero values only within a finite part of the xy plane, then moments 
of all orders exist and the following uniqueness theorem applies [13]. 

Uniqueness theorem: The set of all order moments f pq 
is uniquely determined by F(a?,y), and vice versa. 

Thus if the domain of F(x y y) is continuous, its reconstruction may be 
obtained from an infinite number of moments. In reconstructing a bi- 
nary matrix from its row and column sums we are constrained to do 
so without any of the cross-coordinate moments. Below we summarize 
our approach to solving this problem, based on the orthogonal moment 
method described by Teague [10]. 
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If F(x, y) is assumed to be piecewise continuous over the basic interval, 
then it can be expressed as 

n*,y) =EE L mn P m {*)P n {y) (3) 

where P m {%) is the Legendre polynomial with normalization 

f +1 2 
J dxP m (x)P m ,(x) = + -6 mm . (4) 

and the orthogonal moments L mn , defined by 

im „ = < 2 "' +i f +i )// f , I ,„p„„)P„( >) „> 

are related to the moments f mn by 
(2m+l)(2n+l) 

^mn = T 2-y 2^ ^'mj^nkjjk (*>) 

j=0 k=0 

Equation (3) enables the reconstruction of the binary matrix. This equa- 
tion is sufficient if we have all the moments f mn up to the order N, nax . 
Below we describe how f mn may be estimated from the two projections. 
Let F(X) y) be the binary matrix that is to be reconstructed. Let A(x, y) 
and B(x, y) represent the column sum and row sum projections of F(x,y) 
expressed in the form of two binary matrices. 

Let f pq <,a pq and b pq be the moments of order p,q of F(x, y), A(x, y) and 
B(x,y) respectively. They are defined as follows 

x y 
x y 

and 

i^^^^y) (7) 

x y 

It can be shown that fo q = bo q and / p o = a p o for any p and q. However, 
all cross-coordinate moments, i.e all f pq where both p and q are greater 
than zero, are missing. These moments must be estimated. Let the 
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dimensions of the three matrices F(x,y),A(x,y) and B(x,y) be U rows 
by V columns. Let 

Z i = Y j y q B{x i ,y) 

y 

c i = , £y qF (^y) i = h~,u (8) 

y 

To estimate we define a,* and 6 ? such that 

Ci + ai = Si i=l,...,F (9a) 

Ci+bi = Zi i=h...,V (96) 

In addition to the 2V equations given in (9), we have an additional 
equation, namely: 

Ci = /Oq = S f° r an V 9 ( 10 ) 

i 

Equations (9) and (10) give us 2V + 1 equations for 3V variables 
{Ci,ai,bi). 

Note that the number of equations is less than the number of vari- 
ables. To solve this we use singular value decomposition (SVD) described 
in [11]. This leads to an iterative procedure that is the core of our shape 
estimation algorithm. 

The value of N ma x is determined by the degree of irregularity in the 
shape of the cross section we want to reconstruct. For typical shapes of 
stenotic coronary arteries the value of iV mGX =10 seems to be adequate. 

IV. ALGORITHM 

Our algorithm for reconstructing a binary image from its row and 
column sums is as follows: 

Step 1. Pick an initial Q < N max ( experimentally iV max — Q is 
about 3 to 5). 

Step 2. Rearrange matrix B(x^y) by interchanging its columns 
such that | S{ - Zi \ is minimum in each column i. 

Step 3. Set up the 2V+1 equations as described in (9) and (10) 
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Step 4. Solve for the missing moments up to order Q. 

Step 5. Use Equations (3) and (6) to obtain the resulting image 
R(x,y). 

Step 6, Calculate the error e of R(x, y). This error is defined as 
e —I a p0 — r p0 | + | b 0q - r 0q | for p,q - 0 to Q where r 0q and r p0 
are the moments of R(x, y). 

Step 7. From R generate two new profile matrices A'(x,y) and 
B'(x,y) such that the column sums of A'{x,y) are equal to the 
ones of A(x, y) and the row sums of B f (x , y) are equal to the ones 
of£(s,y). 

Step 8. Replace A(x,y) and B(x,y) by the new A'{x,y) and 
B'{X) y). Iterate from Step 3 until the error cannot be minimized 
further. 

Step 9. Increment Q. If Q is greater than N maT , go to step 10. 
Otherwise, go back to step 3. 

Step 10. Optimize the resulting image against the original row 
and column sums by the following procedure: 

(i) From R{x,y) create the two profile matrices A' and 
B 1 as in Step 7. 

(ii) Calculate the row sum errors of A 1 and column sum 
errors of B { If these errors are equal to zero, exit. 

(iii) Merge A! and B 1 ( by ANDing each pixel of A! with 
the corresponding pixel of B 1 ) to create a new matrix R'. 

(iv) Replace R by R' and iterate from (i) until the errors 
cannot be made smaller. 

V. EXPERIMENTS 

V.l Synthetic data The reconstruction algorithm was tested on syn- 
thetic binary images representing commonly encountered shapes of coro- 
nary arteries, both healthy and stenotic: elliptic, crescent and star-like. 
The results are given in Figure 1. In each figure, (a) is the image to be 
reconstructed, and (b) is the reconstruction. Note that for the ellipse, 
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Figure 1: Synthetic data 




(a) 

Figure 2: Biplane images 
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Figure 3: Reconstructed cross section 

the algorithm gives a reconstruction which is elliptical but the eccentric- 
ity and the orientation of the minor and major axes are wrong. This 
defect can be avoided if we are given additional information such as the 
3-D orientation of the artery. 

V.2 Segment of IN VITRO heart 

For our experiments on a segment of an in vitro heart we obtained 
angiograms at the Mayo Clinic in Rochester, Minnesota using General 
Electric Fluoricon 300 biplane AP and L AT x-ray machine, with tungsten 
target, having a focal spot size of 1.0 mm and a Bucky grid. The set-ups 
for the two views differ. For the AP view, the x-ray source provides 
100 ma, 60 kvp, with an exposure time of 0.003 sec. The distance from 
the focal spot to the object is 57 cm and the distance from the object 
to film is 23 cm. For the LAT view, the power is 50 ma, TO kvp, and 
the exposure time is 0.005 sec. The distance from the focal spot to 
object is 57 cm and the distance from the object to film is 23 cm. We 
used KODAK T-MAT L films, 24x30 cm. The film was digitized on a 
MASCOMP 5700 computer using a Grinnell digitizing package with an 
RCA video camera with a 25mm Inns. Figure 2 shows a biplane pair of 
a portion of an in vitro heart whose arteries are filled with barium* The 
three pins serve as fiducial points. 

Figure 3 shows the reconstruct rd cross section, at the stenosis marked 
by the white line in Figure 2. The asymmetric shape of the reconstructed 
cross section suggests the presence of arteriosclerotic plaque in the lower 
right quadrant of the figure. This and other regions of the coronary 
arteries are in the process of being examined microscopically. 
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VI. SUMMARY AND CONCLUSION 
We have presented a method of estimating the cross-sectional shapes 
of coronary arteries from two simultaneous orthogonal projections. This 
method is based on a new technique for reconstructing a binary matrix 
from its row and column sums. Preliminary tests of this technique on 
synthetic data and on biplane images of a segment of an in vitro heart 
yield encouraging results. 
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Automated Analysis of Heart Size in Digital Chest Images: 
Effect of Heart Size Parameters on Detection of Cardiomegaly 
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Cardiac size is an important and useful diagnostic parameter 
in chest radiographs. However, there is a large variation in the 
subjective judgement of cardiac enlargement, and a confident 
distinction between normal and abnormal can be difficult. In order 
to reduce this subjective element, we are developing a semi- 
automated system for quantitative analysis of heart size in digital 
radiographs. Boundary points along the mediastinum are 
determined by detecting the points from an analysis of edge 
gradients in two orthogonal directions. Heart boundary points are 
selected, and fitted to a "shift-variant" cosine function, to predict the 
entire cardiac outline. We also detect edges of the rib cage and 
diaphragms, and several parameters related to the heart and thorax 
are determined. 

Four hundred chest radiographs were reviewed by three 
radiologists specialized in chest radiology and were classified by 
consensus into two groups: those with and those without 
cardiomegaly. These radiographs were used to construct a 
preliminary database for determination of cardiomegaly. Numerous 
parameters related to heart size were obtained from the semi- 
automated analysis of these radiographs, and the utility of each 
parameter for the detection of cardiomegaly was evaluated by means 
of ROC analysis. We examine whether the accuracy would be 
improved by applying multivariate analysis to a pair of parameters. 
From the analyses of the individual parameters, the cardiothoracic 
ratio was found to be the single most effective measure for detecting 
cardiomegaly in chest radiographs. However, multivariate analysis 
provided results superior to those obtained with an individual 
parameter. 
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An index of the likelihood of cardiomegaly was derived from 
the parameters of cardiothoracic ratio and cardiac area. An observer 
study was performed in which 12 radiologists were presented a 
random sample of 90 chest radiographs with cardiomegaly and 90 
without cardiomegaly. Between two sessions, each observer was 
presented each radiograph with and without the computer generated 
index of cardiomegaly. ROC analyses of these individual 
radiologists using their own subjective criteria, and also as modified 
by the results of the automated analysis of heart size, are compared 
with the results of the automated analysis alone. 
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ABSTRACT 

The total cortical volume is potentially a valuable index of the degree of bone loss 
in osteoporosis. Cortical width was used in prior years as an index, however it was 
poorly reproducible and represented only a very small fraction of the total body bone 
mass. The cortical volume of the hand represents a much larger percentage of total 
body bone mass, and is contained in a single radiograph. 

Digital radiographs make it possible to use a computer to assist in computing 
accurate cortical volumes. The technique is to first outline, i.e. contour or segment, 
the external phalanx and metacarpal boundaries, so that counting the pixels enclosed 
in the boundaries gives the total projected hand bone volume, (the cortex plus the 
medullary space). Outlining the internal cortical boundaries and counting enclosed 
pixels gives us the volume of the projected medullary space. The difference of these 
values is the estimated cortical volume. 

In order to make the use of cortical volume a cost effective procedure, it is necessary 
to automate the segmentation and volume computation of the hand bones and 
cortical boundaries. We have developed an interactive environment for extracting 
intercortical boundaries, and are in the process of developing automated algorithms 
based on current computer vision and artificial intelligence techniques. 

CORTICAL VOLUME MEASUREMENT 

Degeneration of cortical bone cell walls is a common symptom of osteoporosis and 
arthritis. Measurements of this degeneration are useful for diagnosis, tracking, and 
treatment of these diseases. Measures include quantitative methods for determining 
bone density and demineralization [Resnick et.al.-81], intercortical width [Forrester 
et.al.-73], and periarticular demineralization [Thijn-86]. Cortical volume has also 
been used [Lusted et.al.-77], but is usually replaced by measures of intercortical 
width due to the time required to manually obtain the estimated volume. Besides 
being labor intensive, manual measurements suffer from inter- and intra-operator 
variation. Levitt and Hedgcock have proposed the application of computer vision 
techniques to automate these measurements in order to provide them more 
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accurately, rcpcatably, and cost-effectively lo diagnosing physicians [Levitt et.al.- 
89b]. We therefore have two goals: to determine the efficacy of computer 
measurements, and to automate their calculation. In this paper we discuss recent 
work toward these goals in the measurement of cortical volume in hand radiographs. 

The cortical volume is defined to be the volume occupied by the bone cortex of the 
phalanges and metacarpals of the hand. Of more interest is the measurement of the 
ratio of the cortical volume to total bone volume, as this measure is inherently less 
sensitive to variations in patient positioning, scanner parameters and geometry. The 
gold standard for this measurement is obtained from the count of voxels in a CT 
image of the hand that is optimized to distinguish bone cortex. One of our goals is 
to determine the accuracy of the approximation of this ratio from cortical volume 
measured in a computed or digitized radiograph. To this end, the phalanges and 
cortical boundaries must be interactively segmented (i.e. contoured) and measured. 
Figure 1 shows results of this process on a Sun 3/160 using software developed in 
Commonlisp and CLOS at Advanced Decision Systems. Figure 1(a) is the original 
on-line radiograph from the Philips Computed Radiography system. Figure 1(b) 
shows the results of interactive contouring of the cortical volume. In this black and 
while reproduction, the boundaries are more evident on the distal phalanges. 

The interactive extraction is a time consuming process, requiring approximately an 
hour to segment the phalanges in a single hand. Furthermore, the results are errorful 
and difficult to repeat, as there is no quantitative basis on which to chose a best 
boundary. 




Figure 1(a): On-line Computed Radiograph 
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Figure 1(b): Interactive Cortical Boundary 
Extraction 



In a radiograph, of course, the entire bone cortex is projected into two dimensions. 
Radiograph pixels are (essentially) sums of voxels along the lines of projection. As 
such, some pixels include energy both from voxels of the cortex and the medullary 
space; thus, there is no "correct" classification of these pixels. However, the pixel 
segmentation is not the end result; it is the volume estimate we are interested in. 
The correct volume can be obtained by counting the fraction of a pixel that is 
proportional to the percentage of cortex voxels that were projected into the pixel 
Although sub-pixel accuracy in segmentation is the ultimate goal, it is useful to do 
segmentation accurate to a pixel in order to compare against human performance in 
interactive segmentation, and to evaluate the potential gain in accuracy from the 
more computation intensive sub-pixel accuracy algorithms. 

AUTOMATION APPROACH 

The key issue is to develop computer vision algorithms that allow a system to 
automatically find the boundaries and axes of phalanges and the cortical boundary of 
the medullary space. The process of finding the phalanges and related tissue 
boundaries is called segmentation. If segmentation can be automated, then the 
coritcal width and volume can be obtained quantitatively by counting pixels and 
multiplying the result by a conversion factor to obtain measurements in square 
centimeters. With the Philips Computed Radiography scanner, each image is 
recorded with a border grid marked in centimeter tic marks, so the conversion factor 
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can also be determined automatically without knowledge of the relative geometry of 
the scanner and patient. With the Dupont Laser Film Digitizer, the scanner position 
relative to the hand must be known or inferred from a standard object placed in the 
field of view of the radiograph. 

In the model-based evidential reasoning or evidential accrual approach to automated 
segmentation, we regard feature values, observed image structures, shape measures, 
etc. as being evidence supporting or denying hypotheses of what anatomy is imaged 
and how it is oriented relative to the imaging sensor [Binford et.al.-89]. Additional 
hypotheses of possible abnormalities (e.g. fractures, calcification, tumors, 
carcinomas, etc.) can also be reasoned about using this approach if appropriate 
models can be formulated The components of this approach are models of anatomy, 
a well-defined space of hypotheses and evidence, and a mathematically sound 
methodology for accruing evidence in support or denial of the hypotheses. 

We take the point of view that our hypotheses are instances of matching the 
imagery data to the anatomy that we have developed models for. Thus, the 
development and construction of accurate computer representations of 2D and/or 3D 
anatomical models that are valid over human populations is a fundamental step to 
realizing this approach. For hands, some population statistics are available in the 
literature for normal distributions of lengths, widths and ratios of phalanges and 
metacarpals [Poznanski-74], [Meschan-75]. These permit us to begin modeling 
phalanges as cylinders with joints and joint spaces modeling appropriate 
connectivity between phalanges. We are enhancing these models to more accurately 
represent phalanges as we gather statistics on the variance of widths along phalanges 
by making measurements in radiographs. When a geometric and statistical model of 
the hand is partially matched to features extracted from a radiograph, we can use 
mathematical projection of the model, anchored by the feature matches, to predict 
where other, finer, anatomic features lie in the image. Then more refined image 
processing operators can be brought to bear to find evidence for the exact location of 
the anatomical features. 

Results of segmentation of phalanges have been shown by Levitt and Hedgcock 
[Levitt et.al.,-89a], [Levitt etal.-90]. A typical example of phalanges segmentation 
is shown in figure 2. In the following we expand on the the image processing 
operators used to extract evidence of the presence of the cortical boundary between 
the bone cortex and the medullary space. 



CORTICAL BOUNDARY EXTRACTION 

The cortical boundary is more obvious away from the ends of the phalanges where 
the contrast between the cortex and the medullary space is more pronounced. 
However, even in the mid-phalanx, the boundary of the cortex can be faint because 
of the radiographic projection. In order to apply model-based techniques to finding 
the cortical boundary, we make the assumptions that the cortex is of uniform 
(although unknown) thickness, t, and uniform density at any one transaxial cross- 
section, although it varies between cross-sections, and that the transaxial cross- 
sectional shape of the phalanx is approximated by a hemi-ellipse as pictured in 
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figure 3. A and B represent the radiographic image intensities projected through 
adjacent neighborhoods of width w. Determining the cortical boundary from the 
radiograph is equivalent to the problem of estimating t for each transaxial cross- 
section. 
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In this figure, A is the projection through the cortex only, while B is a projection 
of both cortex and medullary space. The expected ratio in radiographic image 
intensity at these locations is the basis for detection of the cortical boundary. We 
chose the neighborhood width, w, such that w is less than or equal to t, because it 
is the gradient across the cortical boundary that we are interested in. Under the above 
assumptions, if the interval [s,s+w] corresponds to the location of A and the 
interval [s+w, s+2w] represents the location of B, then the ratio of radiograph 
intensities, A/B, is given by 



where a is the width of the phalanx and b is the phalanx depth, t is the cortical 
thickness and w is the half-width of the neighborhood over which the ratio is being 
calculated 

Notice that we choose s and w, and a and b are either taken from population 
estimates for phalanx measurements, or a can be read from the radiograph and b can 
be estimated (e.g., if a hemispherical cross-section is assumed, then a=b). With this 
in mind, t becomes the only unknown parameter. 

In our model-driven approach to tissue segmentation, we hypothesize and test 
different values of t by comparing the prediction (above) against the ratios obtained 
in the radiograph. Note that we can find the transaxial cross-section automatically, 
because we can already segment the phalanges themselves and locate the 
longitudinal axis [Levitt et al.-89(a),90]. We then use this as evidence in the 
probabilistic accrual for automatic segmentation. Figure 4 shows the result of 
localized cortical boundary extraction in a radiograph of an arthritic hand. 



Cortical volume is a useful indicator in diagnosis and treatment of arthritis and 
osteoporosis. The volume can be approximated by measurements in digital hand 
radiographs. We have developed a computer environment to perform these measures. 
However, this is a labor intensive, errorful process that renders the measure 
insufficiently cost-effective to be used in clinical applications. 

We have developed an approach to automatic segmentation of tissues of the hand 
that utilizes model-based evidential reasoning to segment demineralized and normal 
hand anatomy in PA views in computed (i.e. digital) radiographs. This allows the 
system to automatically obtain measures of cortical volume and intercoritcal width. 



r(s,t,w,a,b)= 





+ twdx 



SUMMARY AND FURTHER WORK 
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A key step in automation is the development of image processing operators that can 
reliably segment the cortical boundary, Initial results of cortical boundary extraction 
were demonstrated. We are continuing to automate the extraction of cortical 
boundaries. In addition, we are beginning clinical studies to quantify the accuracy of 
the measures in radiographs against the CT measurement of cortical volume. 




Figure 4: Localized Cortical Boundary Extraction 
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Abstract 

We have developed an experimental environment for multi-modal 3D 
reconstructions such as CT, MR, and PET which aids 3D dissection. Due to the 
model-properties of the used XCE model we are able to integrate the results from 3D 
therapy planning procedures such as radiation therapy planning, stereotactic biopsy 
planning and viscerocrariial investigations into the model and, thereby, the 3D 
reconstructions. 



Introduction 

Over the last decade 3D reconstruction and/or visualization algorithms for 
radiological slice data have been developed. They result in some cases in a high 
degree of realism /Herman86, H0hne87, Toennies87, Drebin88, Levoy88A Some 
authors do not concentrate on visualization but on therapy planning. While the early 
systems were only 2D the more advanced developments use common CG/CAD 
techniques as a planning tool /Aldinger89, Becker89, Schmitz89/. Several atempts 
were made to do true 3D therapy simulation with more complex functions for the 
therapeutical planning procedures /Goitain83a, Goitain83b, Toennies89, Tronnier89, 
Fujioka89/. This could be achieved by development of tools and handling functions 
as they were used in the operating theater and/or the therapy equipment. The 
presented system, decribed here from a technical point of view, was specified by 
interdisciplinary cooperation of surgeons, radiologists and computer scientists. 

3D Modelling, Reconstruction and Segmentation 

An experimental environment was built in an extendable form which is based on the 
XCE model /Stiehl87, Tronnier89, Tronnier90/. This model is an extention of the 
well known sell ^numeration model used by several authors /H8hne87, Levoy88, 
Drebin88, Jackfel87/ in a three fold way: First, the voxels have nonisometric 
properties: their dimension in z are different from their dimension in x and y. Second, 
the dimension of every voxel is assumed to be greater than one (and less than three 
due to memory restrictions). Each dimension represents a different property, e.g. CT 
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values, MR signals, segmentation information, results of surgical planning 
procedures and has a distinct information depth. Third, due to the strong need to 
structure the data a scene graph is associated to the XCE scheme via the 
segmentation information. Switching between the two is possible cither by direct 
addressing the scene description using the voxels properties or by associative 
addressing of voxels from the graph side. The graph represents segmentation and 
user supplied information created during a dissection process as well as information 
generated by the therapy simulation procedures (figure 1). 



XCE cube voxel scene graph 




Figure 1: Representation of the XCE model! and the connected scene graph 



Having the slice data converted in our voxel representation 3D reconstructions 
may be created directly by multimodal thresholding, VOI, and 3D-cutaway 
definitions. They can be rendered with transparency for a more comprehensive 
display of topological relations. However, the main task is not reconstruction but 
dissection. It is based on these 3D reconstructions (in particular these reconstructions 
were only 214D) and appropriate conditions as well as an interactive 3D region 
growing algorithm. For the whole process, which we call "non invasive preparation", 
a free configurable volume Tenderer and a scene graph administrator was 
implemented. Both were based on the same set of conditions for selecting voxels. 
They generate 2V£D pictures from the 3D scene in 6 minutes in an draft form while 
segmentation works in near real time on a SUN 4/330 (figure 2). More sophisticated 
visualizations require 10 times of the computational costs by a radiosity-likc 
algorithm. 

3D Therapy Simulation 

Based on the graph description, the XCE model and the 2V£D pictures therapeutical 
planning can be done by simulation. Three application fields were evaluated: 
radiotherapy planning, stereotactic biopsy planning and simulation of max illo- facial 
osteotomy. Simulation results were automatically integrated in the scene description 
by a set of graph theoretical operations. They can be rendered by the same ray 
casting algorithm as described above by specifying some additional parameters. 
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In the case of radiotherapy planning simulation leads to calculation of energy 
distribution. It can be solved by perspectively casting "gamma" rays from a virtual 
energy source into the XCE volume according to the geometrical relations of the 
therapy equipment (e.g. point of gamma source, collimator shape, patient position). 
Along the rays dose values were calculated by using voxel specific absorbtion factors 
derived from the HU values. In conjunction with ray casting this gives the ability to 
consider path length and internal anatomic structures. In this way anatomic and 
simulation data can be rendered in one image based on voxel data (figure 3). 

In our opinion the ray casting approach for dose calculation has two 
advantages: First, the algorithm is very similar to the visualization counterpart and, 
therefore, may be integrated with it. Second, using numerical integration of the voxel 
attenuation rather than calculating the gamma ray's decline results in artificial 
simulation film patterns for the patient's position in the therapy device /Tronnier90/. 
Since less storage access is needed we hope to reduce calculation time by doing three 
tasks in one pass. More sophisticated algorithms taking scattering, ionisation and 
secondary electron emmision into account are known /Sack87/ and may substitute 
our simple dose calculation in a later pilot implementation. 

Biopsy planning is the second field of our research. Based on 3D 
reconstructions of the skull, the brain vessel system and the segmented pathologic 
regions the surgeon first defines a target point to anchor the biopsy needle and then 
turns the needle freely around this anchor point until a good choice is found. As 
shown in figure 4 collisions of the needle-defined "cone of risk" with the vessel 
system can be detected automatically. In any case the system tests the surgeon's 
needle way against the properties of the stereotactic frame system, presented in 
/Rhodes83/, and finally yields the five stereotactic parameters from which the user 
has to supply only three: the target point and two angles of the needle pathway. 

We plan to enhance the biopsy planning aid by introducing a "needle's eye 
view" function which allows the surgeon to estimate distances between the path of 
the needle and surrounding vessels. But during needle motion this functionality 
definitely requires real time volume (scan) convertion of the "cone of risk" and 
rendering capabilities of the underlying visualization system. 

Planning of viscerocranial surgery is an application field with strong 3D needs. 
Our idea is to supply the surgeon with a "box" of familiar tools and handling 
functions which he knows from his daily work: drills, curettes, mills and saws as 
well as handcrafts for bone readjustment and aids for implant design. Using such a 
set of tools the surgeon may simulate the investigation considering all topological 
information in 3D. 

From our technical point of view we classify eroding, grafting and 
transformation functions. Eroding tools require the determination of the intersection 
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of the XCE voxel set and the geometrical shape of the tool's working point and 
trajectory. Grafting functionality requires either the use of techniques for free form 
surface design, mirroring operations or, the so called, off block modelling by the 
above mentioned eroding functions. Again, in either case, the quic volume (scan) 
conversion algorithms seem to be the problem. This can also be applied to the last 
group which is specified as the transformation functions. Affine transformations of 
voxel sets require a significantly higher computational effort than the geometric 
models. This is mainly true due to the greater set of objects to be transformed. But, 
moreover, this is an effect of error minimization neccessary due to partial volume 
effects of the nonisometric discrete transformation. 

Interaction 

Not only the modelling algorithms have to be adapted for use with medical data but 
also the interaction techniques. One problem seems to be the design of suitable 
interaction techniques for deriving algorithmic parameters in 3D from 2V4D images 
by 2D interaction. We use a technique with stcked computational effort because we 
cannot achieve real time while using 3D voxel modelling: The surgeon graphically 
describes his task based on a 3D reconstruction with geometrical tools (e.g. drill, 
working point and direction as well as diameter and depth, see figure 5) and the 
system generates some fast previews and slice reconstructions of the expected 
results. Using this information the tools application can be initiated. It results in a 
volatile XCE description. Based on this representation 3D reconstructions of the tool 
application may be generated. If they seem to be convenient the volatile information 
could be freezed up and included into the scene graph representation. 

Other interactive tools than the already mentioned drilling tool were more 
difficult to implement: Saws may work planar or in a free form fashion (like a pad 
saw). The latter requires tactile feedback between a 3D cursor and/or interaction 
device and the user. The user has to sense if his tool is "on" the surface. The program 
needs not only knowlegde about the working point but also the 3D orientation and 
the user's pressure. This is especially true for tools like curettes and mills which have 
to calculate something like penetration depth. Figures 6, 7 and 8 show a planar saw 
and the results of the sawing and milling operation. 

Interaction techniques for transformations were based on the calculation of the 
segments principal axis. It results in a local coordinate system which is integrated in 
the 3D reconstruction in order to give the user an orientational help facillity. Based 
on overlays like the one showed in figure 9 and a 3D interaction device for inputing 
angles, the local system may be repositioned in place and orientation. Herewith the 
display of the transformed unit vectors of the local system gives the user an 
impression of the resulting segment transformation because real time segment 
transformation and display are not achievable at the current state using voxel models. 
In this way figure 10 of an readjustment osteotomy was created. 
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Creating interaction techniques for free form bone grafts seems to be the most 
difficult task. Common CAD systems would accomplish this but we think they are 
not adequate for users in the field of medicine. Our aim is to support them with a 
maximum of information derivable from 2V&D pictures. This is done by a fast 
calculation of depth reconstructions along the free form grid and the grid itself. This 
gives the user a hint of the 3D extention, shape and bone fitting of his implant. The 
main prerequisites are the real time facilites of rendering during interaction and the 
suitable 3D interaction devices like a 3D tablet. 

Discussion 

Generating sophisticated renderings of voxel data is a well known task but there are 
two problems. First, creating high quality renderings requires high computational 
effort. Therefore we have to be content with draft quality during interaction and 
production quality pictures for documentation purpose. Second, rendering 
paramenters are too formal for convenient use by physicians. We use a tree like 
request structure for rendering and segmentation which is formal enough for 
algorithmic evaluation and is suitable for user interaction because of its easy 
semantics and graphical representability. However, the problem of dissection 
remains. 3D connected component labeling seems to be a convenient tool because of 
its efficiency and easy semantics. The latter is most important when used by people 
not familiar with the mathematical background of segmentation algorithms. Another 
easy useable segmentation tool may be cluster analysis specially for 
multidimensional data. Nevertheless, manual controlling of dissection is necessary. 
The concerning problems of full automatic approaches are often underestimated. 

At the moment there is one major disadvantage of biopsiy planning: 3D vessel 
reconstructions are rarely available. But there are two hopeful approaches: first, 3D 
reconstructions of multiplanar digital angiography data and second, the use of 
magnetic resonance angiographic slice data. The first may lead to higher resolution 
and therefore to more precise planning. It has the disadvantage that it is represented 
in most cases in a geometrical form which cannot be integrated directly into the 
current modelling system. These disadvantages occur not when using magnetic 
resonance angiography data. But, the resolution of that material is still low therefore 
advanced volume rendering techniques should be applied for visualization. 

Viscerocranial planning not only needs topological but also geometrical 
planning methodes. Functions to extract the necessary geometrical information from 
the XCE representation should be provided. Although the total processing time 
which can be achieved could be reduced to IS minutes after CT aquisition, realtime 
capabilities of the planning tools were quite a problem. But we hope to avoid this 
bottleneck by some program optimizations and the use of the next 40 to 100 MIPS 
workstation generation. 
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SUMMARY 

Methods were developed for the registration of gated three-dimensional MR 
and SPECT cardiac images. Left ventricular surfaces were automatically detected 
in the end diastolic frame of both data sets, and the best translational and rota- 
tional transform between these surfaces was determined. The transform was 
applied to the original SPECT image. The registered data sets were displayed 
by overlaying the gray scale MR image with hues related to the intensity of the 
SPECT image. 

INTRODUCTION 

Multimodality image fusion is a tool for combining the information from 
one image with the information from another. Two images of the same object, 
but with different information content are globally matched, or registered, so that 
the structures within overlap exactly. The data in both can then be better inte- 
grated and compared. 

In medical imaging, often the information in one picture is primarily 
structural, while the other picture contains functional data. Registration of the 
images allows structure to be correlated with function. In this study, we used 
magnetic resonance (MR) images that contained high resolution information 
about left ventricular (LV) structure and motion, and single photon emission 
computed tomographs (SPECT) that contained lower resolution information 
about myocardial perfusion. When two such images are registered, the exact 
anatomical location of a perfusion abnormality can be identified, and structural 
effects can be observed. 
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One approach to the image fusion problem has been the use of external 
fiducial markers that can be seen in both pictures. The markers may be auto- 
matically or interactively identified, and the linear transform between the two 
sets can be determined. Similarly, internal identification points may be found in 
each image and matched with each other, e.g., [1]. 

A second approach is to find entire object boundaries in the images and to 
register these. The registration techniques vary; recently published methods 
include principal axes [2], cost minimization [3], and elastic matching [4], The 
direct approach of user-interactive registration of the images themselves has also 
been attempted [5]. 

Most of the previous work has focused on neurological images. Because 
the brain does not move appreciably within the skull, external fiducial markers 
or the outer skin surface boundaries may be matched with each other. Cardiac 
images pose a special problem, however. Registration of external skin markers 
or surfaces does not guarantee registration of the heart within the body, since its 
position changes with body position (e.g., prone or supine), respiration, and 
cardiac contraction. Therefore, the hearts themselves must be specifically reg- 
istered. The following sections describe one approach for performing this task. 

METHODS 

Four patients were imaged with MR spin-echo sequences obtained with a 
repetition time equal to cardiac cyle length and an echo time of 30msec. A 
repeated multislice or rotation technique was used to obtain images in each of 
10 slices at 5 different points in the cardiac cycle. The resulting image resolution 
was 1.7mm in the imaging plane; slices were 10mm thick. Two SPECT studies 
of the same patients were obtained. The patients were injected with Tc-99m 
methylisobutylisonitrile (MIBI) following treadmill exercise, and also during rest. 
MIBI is a new myocardial perfusion agent whose radioactive properties permit 
gated SPECT. In this case, sixteen time frames per cardiac cycle were obtained. 
Reconstructions were created with filtered backprojection using a Butterworth 
filter of order 4 and cutoff of 4.2mm. The resolution of the resulting recon- 
structions was ~5mm in each plane. 

We applied an automatic surface detection algorithm to both the MR and 
the SPECT images [6,7]. The output of this surface detector was a set of 288 
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points on the endocardial surface of the LV for each frame of the cardiac cycle. 
The points were triangulated to create closed polygon representations of the LV. 
The end diastolic time frame was determined using volumes computed from the 
polygons. 

Spatial registration was accomplished by matching the detected LV 
boundaries at the end diastolic frame. The matching operation minimized the 
average distance between the two surfaces over x,y and z translations and rota- 
tions. First, the two surfaces were normalized to the same scale using their pixel 
sizes. They were also registered by their centers of mass. Then, for each point 
(a) on the SPECT surface, a line was drawn through that SPECT point and the 
center of mass (c) of the MR LV. Such a line intersects the MR LV at two points. 
Because the LV is approximated with triangles, the intersection (b) of the line ac 
and the triangle nearest to a can be found. Also, the distance from SPECT point 
a to the triangle intersection point b can be computed; this was considered to be 
the distance between the two surfaces at a. Figure 1 depicts this operation 
graphically. 



Figure 1. SPECT LV surface is depicted as the set of points on the left. MR 
surface is the triangulated surface on the right. The distance between the two 
surfaces at SPECT point a is the distance along line ac from a to b. 
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The average distance over all SPECT points was computed. This distance 
between the two surfaces is a function of x,y,z translations and rotations, and its 
minimal value should occur when the SPECT image is translated and rotated to 
match the MR exactly. We used the iterative technique of [8] to minimize the 
distance function. Iterations were stopped when the function cost decreased by 
less than 5% from the previous iteration. 

A linear transform was computed from the translations and rotations and 
applied to the end diastolic SPECT image. The results were displayed concur- 
rently by overlaying the MR with the SPECT. The gray level of the MR was used 
to define the intensity for each pixel of the composite image. The gray level of 
the SPECT was used to define the hue for each pixel of the composite image. 

RESULTS 

For the seven cases, the average distance between surfaces after registration 
was 2.5mm; the minimum and maximum were 1.9 and 2.9mm, respectively. 
Average time for all iterations was ~5min. One case is shown in detail in Figures 
2-4. Figure 2 shows the two surfaces prior to registration. Figure 3 shows the 
surfaces after registration. Note that in both of these figures, the surfaces have 
already been scaled and registered by their centers of mass. The fusion operation 
is shown in Figure 4. From left to right, the MR, SPECT, and combined images 
are shown. Note the perfusion abnormality seen on the lateral wall of the LV in 
the SPECT image and the marked thinning of the myocardium at the corre- 
sponding point in the MR image. 

DISCUSSION 

This technique follows that of [3] closely. However, the cost function which 
they described requires the surface points to lie as contours in parallel planes. 
The coordinate system that we used for edge detection does not guarantee this. 
Instead, we implemented a cost function that was more appropriate for our 3-d 
surface data. We also coupled the registration process with automatic surface 
detection. 



Figure 2. Relationship between 
SPECT (red) and MR (green) LV 
surfaces before registration. 



Figure 3. Relationship between 
SPECT and MR LV surfaces after 
registration. 




Figure 4. One slice of a cardiac MR image (left). Corresponding slice from the 
registered SPECT image (center). Fused image combining the MR and SPECT 
(right). 
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The registration errors that we obtained in this study were in most cases 
larger than those seen in reports of neurological fusion techniques. This can be 
attributed to a number of factors. The surface detection must be accurate, or the 
distance function will not necessarily reach a minimum when the two images are 
optimally registered. Point spread function and partial volume effects degrade 
SPECT images of thin structures such as the myocardium more than large 
structures such as the brain. Therefore, LV surface detection is more likely to 
be inaccurate. The calibration of pixel size must be correct, for images cannot 
be rescaled to improve registration. Scaling could be added to the minimization 
only if temporal resolution were improved enough to trust that size differences 
between SPECT and MR LVs were not due to mismatches in the cardiac cycle. 

Nevertheless, the quantified errors in the seven studies considered are 
encouragingly low. The fused images also appear qualitatively accurate. 

CONCLUSION 

We conclude that this technique is a useful approach to three-dimensional 
image registration. With this method, cardiac function and structure can be more 
easily and more accurately correlated and compared. 
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Since the introduction of the Apple Macintosh Computer, large numbers of 
users have experienced the ease of a "point and click" windowing system. It 
now seems that overwhelming demand by users is prompting an industry shift 
toward graphical user inter faces. We have developed a graphical user interface 
for our existing image processing software. 

For the past ten years, we have been involved in the creation and use of 
software modules, known collectively as qsh. A descript ion of the image formal 
used by qsh and its use in application programs has been pub lis hod, [1] This 
work is similar to what the later Utah Raster Toolkit[2] has provided for eight 
bit gray scale and twenty- four bit real color raster images. Qsh applies to a 
broader range of images and can be easily adapted to any standard format- 
which is adopted in the future. 

The format of images is a very important consideration, because each device 
that needs to manipulate them needs to know the file format. Additionally, 
it would be convenient if the file format chosen could also act as a database 
for the data associated with the image. Wc developed qsh with both of these 
issues in mind. 

An adaptation of the AAPM standard image format [3] was used for the 
image file structure and the University of North Carolina (UNC) vsh (visual 
shell) [4] was used as a model for developing the image analysis software. Qsh 
is a hardware independent system, written in C and running under UNIX 1 
(and VMS 2 ) which routinely handles the standardization of image data across 
modalities as well as across manufacturers' diverse for mats. [5] 

The image file format, assocated with qsh provides: 1) a header file in text 
file form that contains sets of key- value pairs for the non-pixel data; and 2) 
a separate file composed of the N-dimensional array of numeric values (image 
data) which comprise the actual pixel values. The keys used are derived from 
the following: AAPM, [3] ACR-NEMA 300,[6] and COST-B2 Nuclear Medicine 
group in Europe. [5] 

The header information (the key-value pairs) are stored as ASCII character 
strings, with a key beginning a line, possibly with leading spaces or tabs, fol- 
lowed by a colon equal (:=), followed by the value (in ASCII) and terminated 

1 UNIX is a registered trademark of AT&T. 

2 VMS is a trademark of Digital Equipment Corporat ion. 
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Pixel Format := 
Bytes per pixel := 
Number of dimensions := N 
Size of dimension [0] := 
Size of dimension [1] := 

Size of dimension [N-l] := 



Table 1: QSH Header: Required Keys 

by a newline character. Table 1 illustrates those key value pairs which are 
essential for reading the image. Two routines, ImagePutValue and ImageGet- 
Value are the only routines necessary to use the key-value pairs. Given a key 
as a string, the ImageGetValue function returns the value string by reference 
along with a status. If the user wishes to interpret this string as a number, it 
can be converted to a number (integer, float, double, . . .) by standard C library 
routines. 

Thus key- value pairs have the same behavior as LISP properties. The key- 
value pairs are unordered, they are created the first time ImagePutValue is 
given the key string, and a key retains the value given by last ImagePutValue 
for this key. 

A server or other process can read values from a header file, answer requests 
for values, updates values, add new key- value pairs and update image header 
files. If the key "Number of dimensions := " is set equal to "0", then only a 
header file is expected. This provides facilities similar to environment variables 
(which many operating systems provide) with the added power of being able 
to update these values and share them between separate processes. Typical 
values which users might want to update and share are "Current color scale" , 
"Background", "Saturation", ... 

In the menu system which we have developed, we have used such a header 
file in conjunction with a database. The database maintains a list of projects 
such as monoclonal studies for colo-rectal cancer, or activation - reactivation 
Alzheimer study. When a particular project is selected, a pre-defined set of 
analysis tools as listed by the header file for that project are available to the 
user. A typical project header file is shown in Table 2. By providing a set 
of default values such as image placement, color scales, background, textual 
labels, . . ., we expedite the analysis process. 

Additionally, the database contains the available subjects for each particular 
project. The images associated with these subjects may be sub categorized by 
modality, i.e., SPECT, PET, MR, CT. All the images of a particular subject are 
grouped together. This is shown in Table 3. The image file headers associated 
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Number of dimensions:^ 
Number image type labels:=4 
Number of screen positions:=2 
Image type label [0]:=SPECT 
Image type Y_Location[0]:=0 
Image type X_Location[0]:=0 
Image type Y2 .Location [0]:= 128 
Image type X2_Location[0]:=0 
Screen position image one[0]:=l 
Screen position image one[l]:=3 
Reverse order flag[0]:=0 
Image type label[l] —STRUCTURE 

Number landmark select labels :=6 

Landmark select labeI[0]:=SPECT-Structure Select 

Number landmark call back labels:=6 

Landmark call back label[0]:=Slct ldmks Image One 

Number warp select labels := 3 

Warp select label[0]:=SPECT-Structure 

Number warp call back labels:=3 

Warp call back label[0]:=Warp Image One 

Number display labels:=ll 
Display label[0]:=View Name 

Number processes:=9 

Process label[0]:=Choose Landmarks -STRUC 

Number ROI label types:=2 
Number ROI labels[0]:=18 
Number ROI labels[l]:=15 
ROI designation[0]:=Basal ganglia 

ROI labels [32] :=rt_hemisphere 



Table 2: A Typical Project Header File 
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%project Hemangioma 

%name Subjectl 

/u2/heman/subl CT 
/u2/heman/sublsp SPECT 
/u2/heman/sublnmr32 NMR 

%project Dementia 

%name Subjectl 

/u2/dem/sublnmr STRUCTURE 
/u2/dem/sublt2 PR02 
/u2/dem/subltl SPECT 
/u2/dem/sublm MARKERS 



Table 3: A Typical Database Entry 



with each image for a particular subject can be used to provide default values, 
as illustrated in Table 4, which can be modified by the user at run time. This 
gives the user a great deal of flexibility. Both the image header values and the 
database can be updated as necessary without the necessity of re-compiling the 
menu system. Once the project and the subject(s) have been chosen, the user 
is presented with menu choices for categories of functions. When a particular 
process is chosen, appropriate windows appear on the screen which allow the 
user to perform the indicated operations using an interactive device such as a 
mouse. 

The menu/window system used in this manner differs from either shell 
scripts or protocols. The latter two are essentially one-dimensional in that the 
order of procedure calls and all the choices therein must be defined beforehand. 
The former, however, is two-dimensional in that the order of procedures can 
be altered as needed and more appropriate choices can be made as the process 
continues. There is no need to halt processing in order to re-define any pro- 
cedures or variables. This can be stated succintly by saying that rather than 
being asked questions in a linear order, all the settings can be altered in any 
order the user finds convenient. 

A general purpose display window, shown in Figure 1 is provided in which 
the user can view images by clicking on slice increments or entering a slice 
number in a text window. Window width and level along with the starting 
position on the display screen can be chosen in the same manner. Background 
and saturation values can be changed with sliding scales, and color scales can 
be chosen from a pop-up menu. Information about the study such as subject 
name, exam date, body section, modality, is read from the image header and 
displayed in text boxes. Information relative to the final display, such as the 
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Pixel format := 8 

Patient Name:=Subjcctl 
Patient 1D:=23456 
Body section := abdomen 
Modality:=CT 

Original View Name:=subjectl 
Pixel size[0]:=15,0 

Color sca!e:=BIack_and_White 

Background(%):=I4 

Saturation(%):=98 

Level:- 1051 

Window width: =251 

Transformation Slice[0]:==4 

Slice Des ig n at ion[0]:= double hem angioma 

Transformation Slice[l]:=9 

Slice Designation [l]:=renal vein 



Table 4; Typical Image Header Files 
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Figure 1: General Purpose View Selection Window 
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chosen color scale, is written to the image header for later recall. 

Another menu selection as shown in Figure 2, allows multiple images to 
be viewed together (such as different modality images, e.g. PET, CT, MR, or 
images taken at different times) in order to select corresponding slices. Color 
scales, etc., can be individually chosen for each slice or chosen for all slices 
at once. Similarly, slice selection can be incremented or decremented (i.e., 
moved up or down through the body section) separately or together. A slice 
designation (such as basal ganglia level) as well as all the other information such 
as slice number, background, saturation, optimal color scale, can be written to 
each image header at will. This can be repeated as often as desired for multiple 
slice selection. 

Another menu selection produces a window which allows the recomposi- 
tion of the display screen with previously selected slices, color scales, etc., for 
various analysis purposes such as region of interest drawing (ROI), oblique 
slice reprojection, landmark selection, image warping and statistical calcula- 
tion. Windows for each type of analysis lead the user through the particular 
steps to be performed, i.e., the tools provided are preselected for each project. 
Figure (3a) shows a display configuration window, while Figure (3b) shows an 
analysis window. 

Physician acceptance has been very enthusiastic. Essentially, the windows 
have the easy-to-use Microsoft 3 Presentation Manager interactive behavior and 
a distinctive three dimensional appearance. 

This system is implemented using the X Window System 4 Version 11, Re- 
lease 3 with the OSF/Motif 5 graphical user interface. We chose Motif because 
of its similarity to the Apple Macintosh and the Microsoft windows which were 
already familiar to our staff. Motif is supported by many computer system 
manufacturers including Digital Equipment Corporation and Hewlett-Packard 
Company. It is completely implemented in C and we expect no trouble in 
adapting our work to other interfaces such as Open Look. Only the X Window 
System display screen need be specified; in this particular implementation, the 
binaries are executed on a SUN 3/180 workstation whereas the user can be 
seated at a SUN workstation, a Hewlett-Packard workstation or an X terminal. 

A display sub-library completely implemented in Xlib (so that frame buffers 
or graphics hardware supporting the X Window System can be used) has also 
been implemented. This is in addition to display sub-libraries which support 
the following displays: BarTech TVN256, TVN512; Grinnell 270; Hewlett- 
Packard 978010, -20, -30; Lexidata 3400, 3700; NeWS; Pixar Image Computer; 
Ramtek 9050, 9400; and Silicon Graphics 2400, 4DG, 4DGT. 
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Figure 3: (a)Display Configuration Window (b)ROI Drawing Window 
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Precise analysis of structures or lesions is required from cerebral magnetic res- 
onance images for statistical studies of possible relationships between structural 
malformation and neuropsychiatric disorders. While individual delineation of 
such image features by human operators is very cumbersome, tissue boundaries 
in magnetic resonance images can be identified as photometric edges in the vi- 
sual representation. Many edge detection filters incorporate a scaling parameter 
allowing a tradeoff between edge localization and error rate. In contrast to the 
point based scale space searching that has been proposed, we have developed a 
novel, whole contour based technique for tracing edges selected at a coarse scale 
into successively finer scales to recover the needed precision. We present a rig- 
orous approach to setting the refinement schedule (quantizing the scale space) 
according to the information redundancy between adjacent filters by defining 
a broadly applicable similarity functional. We present experimental results on 
real data and outline other potential applications. 

I. Introduction 

In view of the ever-growing applications of magnetic resonance imaging to non- 
invasive examination of internal structures [1, 2, 3], the need for objective, 
accurate structural determinations and image analysis is apparent. In detailed 
analyses, of subtle features, manual methods become tedious and imprecise. 
Applying computer vision to MRI images may prove expedient in assimilating 
the vast amount of data. A qualitative connection between schizophrenia and 
enlarged ventricles [4] has been found. To quantify this, one needs to acquire a 
large amount of data for a statistically significant conclusion. This is very slow 
and inefficient if done manually. 

The most useful semantic representation of a magentic resonance image is a 
description of the tissue boundaries, which may be detected as photometric 
edges using, for example, the Lap lacian-of- Gaussian (LoG) operator. The LoG 
has a scale space constant, <r, which determines the degree of edge resolution. 
For relatively large values of cr, the edges depict the image contours in a coarse 
manner. Although the significant contours are represented without much su- 
perfluous detail, the existing edges provide only a rough, imprecise outline. As 
cr is reduced, edge accuracy improves, but this is accompanied by an increased 
number of spurious edges, cluttering the image. We require an algorithm in- 
corporating both the compactness of a coarse scale representation with the 
precision of a fine scale edge map. 

The motivation for tracking edges through scale space stems from the premise 
that, as shown by [5], pixels do not migrate further than two units per unit 
change in scale parameter. Bergholm proposes a point-based algorithm for 
"edge focusing" , resulting in contours at a given stage representing a refinement 
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of the previous image within 1-pixel windows around all edge points. This 
method, however, leads to unnecessary noise in the final image because all 
edge segments near old edges are kept, regardless of their correspondence to 
old edge segments. Also, the migration rate assumption breaks down at very 
fine scales: pixels move more rapidly than predicted. Bergholm sets Ac = 0.7 
for <7 > 2 and Ac = 0.3 for c < 2, but he does not employ a rigorously defined 
schedule. 

We present an algorithm which automates the process with high accuracy. The 
algorithm refines the boundaries of the important brain structures, detected 
at a coarse scale, by following their progress through the scale space, enabling 
us to delineate them very precisely. Ours differs from other edge refinement 
algorithms in that it considers entire contours, not their constituent pixels, 
giving increased robustness in the presence of noise. The refinement schedule is 
also established from rigorous theoretical underpinnings. Lastly, this algorithm 
could allow the user to focus on a few contours of interest, since a practical 
image analysis tool would probably incorporate some sort of contour selection 
capability. 

Successively refined edge representations in scale space correspond to increas- 
ingly accurate depictions of the actual contour. So for a given contour, the 
edge description at one level of resolution will not be dramatically different 
from that of the same contour at a slightly finer scale ([6]). Although any par- 
ticular point may fluctuate rapidly, the net effect is minimal. Coarser regions 
of the scale space produce smoother contours, which are locally more nearly 
straight; thus, greater detail at finer scales requires more pixels to represent 
that detail. In view of this, we choose to follow sets of points, or contours, 
instead of individual pixels. 

One might consider an attribute-based method, but this is inadequate because 
it lacks robustness and it fails to handle split and merge events. The algorithm 
we present uses only the property of limited pixel migration, and no explicit, 
whole contour attributes. The focusing routine compares contours from one 
level in scale space to those in another by directly calculating a "degree of re- 
latedness" between them. This nuihber is based on a normalized difference in 
pixel positions, where consensus built from the votes of individual pixels deter- 
mines how well the two contours match one another. If the degree of relatedness 
is sufficiently large, then the contour at the finer scale must correspond to the 
contour at the coarser scale, and the new contour is kept. The final prod- 
uct is an image consisting only of fine scale contours directly descended from 
(selected) initial coarse scale edges. 



II. The Similarity Functional 

In edge refinement we have to go through the scale space from coarse to fine, as 
quickly as possible with the fewest possible errors. For any matching process, 
one must choose a next scale of the edge operator such that the contour gross 
shape is relatively stable and individual edge point migration is not excessive. 
If we take very small steps then the contours will not move much, but the 
process of focusing will take a long time. A very large step can introduce large 
changes in the contour, requiring a complicated, inefficient matching process. 
Our matcher, while accommodating splits and merges, nonetheless assumes 
the degree of individual edge point migration between scales to be limited. 
If the edge detectors at two scales are similar, their responses, i.e. the edge 
contours, will also be quite similar, and vice versa. So we define the similarity 
parameter, (E), as afunctional which measures the expected degree of similarity 
in the output of the edge detector at two different scales. It is based on the 
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cross spectral density of the response at two different scales. This similarity 
functional then sizes our steps through scale space. 

Let the edge operator impulse response at two different scales, a\ and ao 
(<Ti > 0-2) have Fourier transforms given by H a2 (f) and H ai (f)> respectively. 
The cross spectral density of the outputs when driven by white noise is given 
by Sb* B+ if) = //^ 2 (/)//^ t (/) where the * superscript denotes complex con- 
jugation. We define similarity by; 



\j SZo Su^h^ (f)df JZo Sh^ (fW 



£(•, », •) is the integral of the cross spectral density, normalized by the square 
root of the product of the spectral densities of each filter. Note that when 
02 — E = 1.0, implying identical responses, the limiting case of similarity. 
When the frequency response of the two edge operators are such that that they 
do not overlap at all, then their cross spectral density is zero and so is the 
similarity. 

When entering the image to be processed, the user also sets a constant value 
for similarity between two levels of a. For high similarity values, e.g. 0.95, 
the schedule will consist of relatively small values for Aa; that is, focusing 
takes place in small steps through scale space. As the similarity specification 
is decreased, the allowable jump size increases, but the potential information 
redundancy falls off. We can also set a schedule of constant step size, Act, 
and anticipate edge migration commensurate with the present similarity value, 
which would then be different at every step. 

A. The Laplacian-of- Gaussian Edge detection operator 

The Laplacian-of- Gaussian edge detector is expressed mathematically as 

V ^ (x ' y) = 2^? (2 " (^T^"^ ( 2 ) 

The LoG is a circularly symmetric operator in both the space and frequency 
domains. Setting S to a constant value, say S, letting a\ > a 2 > &i — &2 = Aa } 
and denoting cri by a, the value of the current scale, we get a relationship 
between the size of the step we want to take at a particular scale and the value 
of the current scale: 

^ = (\ _ *^-vp^ J kLoG = {2/si) (3) 

Equation (3) relates three parameters: the present scale, a r the next scale step, 
Act, and the similarity, S. In matching we can fix either of two parameters: Aa 
or 5. The starting value of a (say <Tq) is also specified as well as some stopping 
criterion. 

If Aa is fixed our matching process has to expect changes according to the 
similarity value, itself a function of the present a. Fig. 1(a) is a plot of the sim- 
ilarity functional versus the present scale. Each curve corresponds to constant 
step size. For a specified step size, the focusing process operates along one of 
these curves. Given the present scale, <r, we have a measure of the expected 
similarity of the filter response at the next scale, a — Act. Our selection of 
Aa in the constant step size approach is constrained by the smallest desired 
scale of the refinement process. We choose the largest step, using the curves 
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in Fig. 1(a), such that the similarity will not go below some lowest similarity 
that can be tolerated by the matching process (typically 0.8 for our algorithm). 
Also, when choosing a starting point, one must determine if: all contours of 
interest are at least coarsely represented, and the image is sufficiently noise-free 
to minimize the perpetuation of unimportant initial edges. The path marked 
with arrows in Fig. 1(a) is one of the schedules followed by Bergholm [5]. Note 
that Bergholm had to take smaller steps after a = 2.0 because the similar- 
ity value would have decreased further making the matching prone to error; 
however, he did not quantify this effect as we have. 

Another parameter which we can specify in Equation (3) is the similarity while 
A<7 varies. This ensures consistent matching reliability. For constant similarity, 
5, the ratio of the step size to the present scale, is constant. This ratio 
is a function of the chosen similarity, giving a second operating characteristic. 
Given the minimum similarity we can tolerate, we select the ratio The 
step size is then proportional to the present <r, which agrees with intuition. 

B. The zero crossing operator of Sarkar and Boyer 

We now develop the scheduling relationship for the better (indeed optimal) edge 
detection operator suggested by Sarkar and Boyer [7]. 
The operator impulse response is given by: 

h y (x) = A f 1 + -\x\ — — J e y - oo < x < oo (4) 

7 is a scaling parameter analogous to u for the LoG. As 7 is increased the 
spatial extent of the filter increases resulting in fewer spurious responses and a 
loss of detail and fine edge structure. Following a similar procedure in deriving 
E as was done for the LoG, we obtain 

* m M.{ia^EE£^ (5) 

This expression is similar to Equation (3); we have analogous operating char- 
acteristics and operating plots similar to those for the LoG can be made. 
It is interesting to compare the curves for the LoG and the optimal zero crossing 
operator. Fig. 1(b) plots the ratio of the step size to the present scale against 
the similarity functional. Note that the curve for the optimal zero crossing 
operator lies above that of the LoG for all values of the similarity functional. 
This means that one can take larger steps across the scale space for the optimal 
zero crossing operator than can be taken for the LoG. 



III. Refined Contour Selection Procedure 

Our matching schedule depends on edge pixel migration being bounded for a 
specific change in the scale. We have noticed that Bergholm's estimate of pixel 
movement is valid only for £ > 0.8 (see Fig. 1(a)). For lower similarities we 
observe greater average pixel migration. 

At every focusing step we maintain an array, called the match- table. Each 
column corresponds to a contour from the smaller scale (a or 7). The rows 
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correspond to the contours at the present scale. Each match table entry is 
the match factor denoting the quality of match between the two corresponding 
contours. Because of the pruning of the search space by the early exit condition 
on the distance summation, there are no entries corresponding to contour pairs 
having a match factor less than 0 .7. The match factor is simply a warping 
(for human consumption) of min-d values into the range 0 — ► 1 such that 0.7 
corresponds to the smallest value for a valid contour match (Bergholm's limit): 
1 is a perfect match, and 0 is an impossible match. 

Fig. 2(a) shows the closed LoG contours detected at a — 4.5 of a coronal MM 
brain image. The same image appears in Fig.2(b), but here a = 4.0, and several 
interesting changes have occurred. In the upper left hand region of the scene at 
<r=4.5, the contour labeled p lies separated from contour a (the rough cerebral 
outline). But in Fig. 2(b), what was p has now merged with the larger contour. 
In total, three contours at the coarser scale (a, p, and c) merge into one, 6, at 
the finer scale. 

Table 3 is the match-table comparing Figs. 2(a) and 2(b). To check if the 
algorithm detects both merges, look under the column labeled "6"; indeed, 
there are three significantly large match-factors. The algorithm has determined 
that old contours a, p, and e are now represented by new contour 6. Also note 
that the corresponding match factors are much greater than 0.7. In the domain 
of pixel migration, this means that average pixel movement along the contour 
is much less than Bergholm's upper bound. This is encouraging because it 
suggests robustness with respect to noise and the threshold selection. 

IV. Experimental Results 

This section presents results derived from coronal magnetic resonance images. 
Fig. 4(a) represents a somewhat posterior section through the brain (note the 
presence of the cerebellum). Fig. 4(b) shows the edges detected for o~ = 5.0, and 
Fig. 4(c) corresponds to the outline at a — 2.0. The edges of Fig. 4(d) are those 
kept from a = 2.0 after performing three edge refinement steps of Aa = 1.0. 
Fig. 4(e) results from keeping the similarity constant (in this instance, E = 0.8) 
and allowing the step size to vary. Based on this, the schedule for a required 
only two comparisons: from 5 to 3.3, and from 3.3 to 2. The resulting images 
are virtually identical. Clearly a constant similarity schedule traverses scale 
space more efficiently than a constant step size schedule. 

The next series of images illustrates how the algorithm works on the same 
gray-level image, but with the optimal operator of Sarkar and Boyer [7]. The 
operator does not give closed contours like the LoG, but can traverse a broader 
range of scale space in fewer steps. Fig. 4(f) represents 7 — 3.0, Fig. 4(g) 
is at 7 = 1.0. Using 7 = 3.0 as the starting point and 7 — 1.0 as the final 
point yields Fig. 4(h) when applying the algorithm. Starting instead from the 
7 = 3.0 picture and removing one-pixel edges yields results in Fig. 4(i), The 
identical nature of Figs. 4(h) and (i) implies that by initially removing one-pixel 
contours, one eliminates a number of computations, making the algorithm more 
efficient without affecting the final output. An advantage of focusing using this 
edge detector is that a great deal of time is saved since there are no extremely 
long contours like those from the LoG. The time spent in distance calculation 
is significantly reduced. 
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(a) (b) 

Figure 1: (a) Curves with constant steps (Aa = 0.1 to 2) in scale space for the 
LoG operator, (b) Comparison of the Ratio of the step size and the present (a 
or 7) vs. the similarity functional for the LoG and the optimal zero crossing 
operator of Sarkar and Boyer. 




(a) (b) (c) 

Figure 2: (a) Closed contours detected with a = 4.5. (b) Closed contours 
detected with <r = 4. (c) Traced edges. 
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Figure 4: (a) The gray-level image of a slightly posterior section of the brain, 
(b) LoG at a = 5.0. (c) LoG at <r = 2.0. (d) Traced edges with constant 
Act =1.0. (e) Traced edges with constant inter-sigma similarity = 0.8. (f) 
Zero crossing edge detection at y - 3.0. (g) 7 = 1.0 Scale-space matching 
starting from (h) 7 = 3.0 and (i) thresholded 7 = 3.0. 
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Abstract. A new technique, complex line moments (CLM's), is developed for 
encoding linear features in images. This technique is analogous to complex moments 
computed from regions. An algorithm is developed for computing CLM's from chain- 
coded boundaries obtained from grey- value images. The accuracy of these features is 
investigated. It is found that this technique is effective for measuring the location and 
orientation (rotation) of an object 

The CLM technique is incorporated in an image registration algorithm which is 
capable of matching objects part of whose outlines are dissimilar. The algorithm can 
compute transformations between images being matched with sub-pixel accuracy. 
Distance transforms of both objects are used to identify portions of images that are 
homologous in a geometrical sense, and CLM's of the homologous portions are used 
to compute the displacement and rotation which will place the images into 
registration. Feature extraction and selection of matching portions is done 
automatically in an iterative algorithm. The algorithm initially operates in a tolerant 
mode, attempting to place all images features into registration. Subsequently, the 
distances between portions of the objects are used to produce increasingly stringent 
criteria for rejecting disparate portions of the two objects, and similar portions are 
registered by computing the appropriate geometrical transformation. 

Experiments were performed with images of autoradiograms of successive 
sections of rat brains. The algorithm is capable of registering images even when parts 
of the outline are outside the field of view, and when the image outlines are distorted 
because of errors in the edge extraction procedure. 

I. INTRODUCTION 

In image analysis, one of the fundamental operation is the comparison of features 
in related images. Some examples include angiography, analysis of image sequences 
(cineradiology), and followup studies of a patient In autoradiography one often needs 
to compare pairs of homologous images (histology and autoradiogram) or 
autoradiograms from serial sections [1]. Most such studies require image registration, 
that is, the mapping of one image to another so that homologous objects fall on 
corresponding coordinates, and this requires the measurement of the mapping between 
the coordinates in the two images. In the simplest case, this mapping is an isometry 
(a combination of a translation and rotation). 

Correlation is the most common technique for identifying corresponding objects 
in pairs of images [2, 3]. Though powerful, this technique is somewhat inflexible and 
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requires enormous volumes of computation. Another strategy is to extract features, 
and to perform registration on properties of these features. One of the most common 
such feature sets are the moments [4], Initially, moments were used as descriptors for 
pattern recognition, though they also have been applied for image registration [5, 6], 
These techniques are quite effective when the object being matched have the same 
shape, differing only by an affine transformation. In general, the problem is much 
harder. Paris of the objects may be identical but other parts may differ: this may be 
caused by occlusions, incomplete views, or failures in delineation. Thus, in general, 
one must also solve the so-called correspondence problem, that is, to identify the 
portions that are homologous. 

In this paper, we describe a technique for two-dimensional partial image 
registration which simultaneously solves the correspondence problem and determines 
the best transformation for preforming the registration. The registration 
transformation is based on complex line moments: these are geometrical moments, 
such as those studied in the literature [4, 7, 8]. Commonly, moments arc based on 
functions of two variables. In our work we compute line moments which are 
evaluated from linear structures such as edges or boundaries. The correspondence 
problem is solved by the use of the distance transform, an efficient technique for 
identifying which parts of two images correspond. We develop an iterative boundary 
matching algorithm, and present experimental results on partial image matching. 
II. COMPLEX LINE MOMENTS 

Assume a curve is represented by the parametric equation (X(t),Y(t)) for a<t<b. 
Another form of this is the complex function Z(t)=X(t)+/Y(t). We define the 
Cartesian moments of the line by 

b 

a 

where 

dt" V 1 dt } n dt ; 
The complex line moments are defined as 

b 

Cpq [Z] = JzP(t)2>? 
a 

where Z(t)=X(t)+/Y(t) and Z*(t) is the complex conjugate of Z(t). Complex moments 
have the following attractive properties: 
(I) Rotation Properties: 

If the image f,(x,y) is rotated counterclockwise through a angle 0 (or the image 
plane axes rotated through a clockwise angle 8) to become f 2 (x,y) so that 
f2(x2>y2Hi(*i,yi) where 

[xi i r cos e sin e i rx2i 
yiJ L -sin 9 cos 0 J LyzJ 



dt 



dt 
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then the new set of Complex moments, namely Cpq is related to the old by the 
equation 

(2) Translation Properties: 

If the image fi(x,y) is shifted by an amount w=u+iv, (or the image plane axes 
shifted by -w) to become f2(x,y), then the new C pq is given by 

c pq = £ E^w*^(p)(;)c p . m><7 .„ 

m=0 «=0 



(3) Scaling Properties: 

If the image fi(x,y) is scaled by a scaling factor (or the image plane axes 
scaled by l/K) to become f2(x,y), then the new C'pq is related to Cpq by the relation 

C pq = KP+<I+ l C p q 

The centroid of the curve can be calculated by wc=C 10 /Coo. The Central Complex 
Moments are defined by the equation 

b 



dt 



They may be computed by first finding the ordinary complex line moments, 
computing w c from Cio and Coo as shown above, and then applying the shift 
property. 

Suppose Cpq and Cpq are the complex line moment of two curves that differ 
from each other by a combination of a rotation and translation. We may represent the 
transformation by a rotation about the centroid followed by a shift. The centroids of 
the two curves can be found from the first- and zero-order moments. The rotation can 
be found from the phase (argument) of the complex moments. For example, 

e = arg(Q 2 i)-arg(Q , 2i). 
If, furthermore, the two curves differ by a scale factor, it can be found from the ratio of 
the zero order moments. 

In our computer programs[9]. curves were represented by chain codes That is, 
the curve consisted of line segments with knots on adjacent pixel coordinates, and each 
chain code element is interpreted as a line segment. We compute its complex line 
moments from the centroid, angle, and length. The total line moments of the curve 
are computed just by summing the line moments of all chain code segments. 
Numerical trials on boundaries of geometrical figures such as rectangles and triangles 
indicate that the chain-code representation can affect Coo by as much as 20%. In spite 
of this, the values of centroid and orientation angle computed from the moments were 
sufficiently accurate to allow sub-pixel registration. We found that best accuracy in 
rotation angle estimates were obtained by examining both Q20 and Q21- 
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ID. MATCHING PRINCIPLE AND ALGORITHM 

If two curves are identical then complex line moments provide a convenient 
technique for finding their registration parameters. The basic idea in our work is to 
find those portions of two curves that are identical, and to use complex line moments 
to register them. Matching portions are identified by using distance functions and 
distance transforms. The basic idea is to find an approximate registration 
transformation, to identify matching parts after registration, and to recompute the 
registration of the matching portions. If we continue to iterate the same process 
several times, the final result will reach some stable or best point which may satisfy 
our criterion* Complex line moments are attractive because they are well adapted for 
partial boundary matching. 

The distance function is a convenient tool for both identifying matching portions 
of two curves and for computing their CLM T s. The distance from a set S to point x is 
defined as: 

d(x,S) = minyes'lx-yll. 
When viewed as a function of x, this is called the distance function (distance 
transform) of set S. In our work we use an efficient algorithm proposed by Borgefors 
[10] that approximates the Euclidean distance for each pixel. The distance transform 
converts a binary digital image, consisting of feature (boundary) and non-feature (non- 
boundary) pixels into a gray-level image where the value of ail non-feature pixels are 
approximately equal to three times the Euclidean distance to the nearest feature pixel. 
Figure 1 shows an example of the distance function computed by the Borgefors 
algorithm. 



Figure 1 (a) The original image 
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(b) The distance function of the image 



Suppose we have two sets: B A, the reference curve and BB, a control curve. The 
distances between points on one curve and the other curve can be easily found after we 
compute the distance functions of both curves. The matching portions BA* and BB 9 
are defined by: 

B A' = { x I x € BA , d(x, BB) < y } 

BB' = { x I x € BB , d(x, BA) < 
where \jf is a matching distance criterion. To decide the proper criterion \|f for rejecting 
the unmatched parts is tricky: if the distance criterion is too small, it is possible to 
reject some matched parts of the boundaries, and matching performance may 
deteriorate. The method used for selecting the criterion y is based on the maximum 
distance from control boundary to reference boundary (MDBA), which is defined by 
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MDBA = max X6 BB d(x,BA). 

A coefficient £ is used to multiply MDBA to produce the distance criterion y> and this 
coefficient decreases slowly in succeeding iterations* The basic scheme is to gradually 
decrease £ in order to avoid discarding the parts that should be considered 

The distance criterion is used directly to compute the so-called weighted line 
moments, defined by: 

C M [BB']= JzajPZ^Oco^OJdt 
BB 

where co^z) is a weighting function: when computing the moments for BB\ 

fl ifd(z,BA)<v 
®iW- \o ifd(z,BA)£y 

The Algorithm. 

The matching algorithm is a technique for finding the best transformation 
between boundaries in two different images. Input to the algorithm are two images 
with thresholds specifying the criterion for extracting boundaries. The effectiveness of 
matching is measured by a generalized distance function that depends on the two 
boundaries and the transformation. The algorithm's outline is given in Figure 2. 

Step 0. INITIAL MATCHING 

Step L IDENTIFY CORRESPONDING PARTS AND COMPUTE 

Step 2. COMPUTE TRANSFORMATION. 

Step 3. EVALUATE TRANSFORMED RESULT. 

If termination criterion is met, STOP. Otherwise, go to Step 1. 

Figure 2. Boundary Matching Algorithm 

Step 0. The two boundaries B A and BB (the reference and control curves) are 
represented by two sequences of chain codes. Complex line moments up to third order 
are computed. The centroids are found from the zero'th and first moments, and the 
second and third order central complex line moments are used to get the rotation angle. 
INITIAL MATCHING consist of a shift and rotation of the control boundary BB 
according the parameters obtained from the line moments. If the boundaries are 
different this will not give the optimal transformation. 

Step 1. IDENTIFY CORRESPONDING PARTS AND COMPUTE WEIGHTED 
MOMENTS. In this step we try to identify the corresponding parts of the control and 
reference boundaries, and these are used when we compute the weighted moments. We 
first compute the quantity MDBA defined above from the transformed BB. This and 
the coefficient £ are used to obtain the distance criterion y. The weighted moments of 
BA and the transformed BB are then computed, as described above. 

Step 2. COMPUTE NEW TRANSFORMATION. As in in initial matching, the 
shift is found from the centroids, while the rotation is based on the second order 
moments. Note that this is the transformation between the transformed control 
boundary and the reference boundary, so that the full transformation is found by 
composing this with the transformation used to modify BB in Step 1. In subsequent 
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iterations we compute the transformed BB by applying the composed transformation 
to the initial control boundary. This is done because transformations can be composed 
very accurately, but iterated transformations of a chaincoded boundary produce large 
numerical errors because of the chaincode representation. 

Step 3. EVALUATE TRANSFORMED RESULT. In this part, our objective is to 
examine the re-matching result and compare it with the previous one. If the new 
transformation does not improve the match, we discard it and go back to adjust the 
distance criterion y trying to match again. If the new result is better than the old one, 
we will store this result and still go back to STEP 1 trying to make the match even 
better. We examine EBB, defined by 

EBB = £d(y,BA). 
yeBB 

We compare the values of EBB before and after the iteration. If EBB decreases, we 
accept this transformation and set the iteration flag Y to zero. If it increases, we 
discard the new transformation and increment the iteration flag. The termination point 
is perhaps the trickiest part of the algorithm. We find that once the error function 
reaches some optimum value further decreases of the distance criterion £ causes some 
of the valid parts of the boundaries to be rejected. The method we use is: if the 
iteration flag Y is zero, ( is decreased and the transformation is accepted. If the 
iteration flag is positive but less then 4 the transformation is rejected but £ is 
increased. The process stops once the iteration flag is greater than 3. This criterion is 
based on experimental results. We found that once the number of vain trials was 
greater than 3 it was unlikely that there would be further improvement 
IV. EXPERIMENTAL RESULTS 

The algorithm was evaluated in controlled trials by applying it to pairs of 
boundaries that differed in vaious ways. Figure 3 shows a example of the whole 
matching process to demonstrate different steps of the algorithm. The reference image 
in Figure 3(b) and the control image, shown in Figure 3(a), have different orientations 
and parts of the object are missing. First pass matching, shown in Figure 3(c), is not 
accurate because of the missing boundary portions. Figure 3(d) is the comparison 
between the first pass matching and the first iteration of the algorithm, and we see the 
result is much closer. In following iterations matching is even better, as shown in 
Figures 3(e), (f) and (g). Figure 3(h) shows the final result 

V. CONCLUSIONS 

The image matching technique based on line moments and distance functions 
proposed in this paper is a new approach to image registration. The work was 
motivated by the need of developing a simple and efficient method to find the best 
matching position of two partial images. The novel results in this paper can be 
divided into two parts: the development of complex line moments, and a new 
boundary matching algorithm. 

Complex line moments enable us to work on the matching problem when only 
parts of the image boundary are available, so that conventional area moments cannot 
be applied. The use of complex line moments in place of area moments has proved to 
be a convenient and reliable way to solve the matching problem. Complex line 
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Figure 3. (left to right, top to bottom) (a) The control image and its boundary, 
(b) The reference image and its boundary, (c) The image after the first pass matching, 
(d) Comparison between the first pass matching and the first iteration, (e) The result 
of the second iteration is better. (0 The result of the third iteration is getting even 
better, (g) After several iterations, (h) The final result 
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moments arc computed from chain-code representations of the boundary, and we found 
that this limits the accuracy. 

The boundary matching algorithm based on line moments shows that line 
moments can be used for partial image matching. Other algorithms based on this idea 
can be developed. For example, the main weakness of this technique is that once the 
matching process starts going toward to wrong matching transformation, it is difficult 
to retreat and find the correct match, It may be worthwhile to implement a decision- 
making expert system to guide the matching process. Some extension of this 
approach are possible. For example, this algorithm can be combined with the 
footprints! 1 1] or landmark recognition [12], or edge and texture detection, which will 
allow this technique to work more effectively and accurately. To reduce the distortion 
of image transformation, a different representation for lines is of interest for future 
research: for example, Bczier's cubic curves may introduce less distortion in image 
transformation. 
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Abstract 

A triangulated 3D representation originates from contours which 
approximate boundaries of segmented objects in CT or MR slices. 
Since contours are the framework for surface reconstruction an 
efficient surface representation depends on efficient contour 
approximation. For that purpose we present an algorithm which 
minimizes the number of approximating line segments for a given 
similarity constraint. The method is based on a graph-theoretic 
approach where the optimal solution is a minimum cost path in a 
graph representing boundary points and possible line segments 
between such points. The optimal contour approximation is compared 
to two heuristic methods for fast contour approximation. The first 
algorithm uses the principle of extending line segments until a given 
similarity constraint is violated. The second algorithm creates a 
contour by subsequent subdivsion of line segments until a certain 
similarity constraint is satisfied. 

Introduction 

3D surface reconstruction by triangulation generates triangular 
surfaces from polygonal descriptions of object boundaries in 
segmented CT or MRI images [2],[4],[7],[9]. Such images are 
discrete, two-dimensional scenes of picture elements (or pixels) whose 
values represent the parameter which is measured and reconstructed 
by the imaging system. A segmentation operator generates a binary 
scene from that image by assigning to each pixel either the value '0 f 
(background pixels) or the value T (object pixels). Contours are 
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created from the binary image by applying boundary tracing and 
contour approximation. 

Boundary detection traces bounding edges of object pixels which 
form connected sets in a binary image. Two object pixels are connect- 
ed if they share at least a common vertex (there are other definitions 
of connectivity which result in slightly different boundary tracers). A 
new boundary is detected in a binary image if an unmarked transition 
from a background voxel to an object voxel is found while scanning 
the image in a line-by-line fashion. The edge at such transition is 
marked and stored as initial element of a new boundary. Beginning at 
the initial edge a loop follows where we search for the successor of 
the current boundary edge until the first edge is reached again. (The 
determination of the next boundary edge is done by looking for a 
background-to-object transition in the four-pixel-neighbourhood at the 
vertex between the current edge and its potential successor.) 

Boundary tracing results in a vertex list describing boundary edges. 
Since the large number of boundary edges would not lead to an 
efficient surface representation a contour approximation step follows 
which removes as many vertices as possible while retaining major 
shape features. 

Optimal contour approximation 

With given vertices V={v l9 V2,...,VfJ of a boundary we wish to 
select a subset V c =t v c(l)> v c(2)>—> v c(m)} which approximates V in an 
optimal fashion. Optimality may be defined either as maximum 
accuracy to be achieved by a given number of elements of V c or by the 
minimum number of elements of V c needed to meet a certain accuracy 
requirement. We prefered the latter since we want to describe shape 
features with a pre-set accuracy and with a minimum number of 
vertices. 

The degree of accuracy is measured by a function dev(ij) which is 
the maximum distance of all vertices between two vertices and Vj 
and a line through v t and Vy. The accuracy of an approximation V c is 
satisfactory if dev(ij) is less than some maximum deviation maxdev 
for all lines <v z v / > of V c . The search for an optimal subset V c of V is 
realized as search for a maximum cost path in a graph whose nodes 
are vertices of V and whose arcs are straight line segments between 
nodes. (In the following we are doubling the set of vertices in V such 
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that v n+J -=v f for i=i,...,n. That simplifies our explanations since the 
wrap-around effect at the two vertices v n and V; has not to be taken 
into account.) 

For arcs representing lines <v / ,v / > with 0<ij<n+l we define the 
cost function 

cost(ij)- ,ifdev(ij)<maxdev 
0 , otherwise. 

Using the cost function above we apply the algorithm in figure 1 to 
search for a maximum cost path starting at vertex Vj. The result is a 
subset V n +j of V with the smallest number of vertices of all subsets 
which include the vertex v 2 and whose accuracy is satisfactory, (The 
accuracy requirement is met, since the cost of arcs is set to 0, if the 
deviation of approximated vertex positions exceeds maxdev.) 

begin 

node-1; 
pathcost^O; 
node=node+l; 
while node<n do 
begin 

pathcost node =cost (l,node) ; 
for i=2 until node-1 do 
begin 

if pathcostt+cost (i,node) >pathcost node then 
begin 

pathcost node =pathcost i+cost (i,node) ; 

end; 
end; 
end; 
end. 

figure 1: Optimal contour approximation 

Since the solution is only optimal for contours containing the 
vertex v 7 , the procedure has to be repeated n~l times using the vertices 
{v2,—>VfJ of V as start nodes. The maximum cost path of all these runs 
corresponds to a contour with the minimum number of vertices 
satisfying our criterion of accuracy. That criterion, of course, can be 
replaced by others to achieve different results (e.g., by the area under 
the approximating polygon line to allow for noise corrections[l]). 
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The computation time of the algorithm increases cubicly with the 
number of vertices in V. Since a boundary in a typical CT scan may 
have more than 4000 vertices, computation times can be unaccept- 
ably high. Therefore we did not apply the method for contour approx- 
imation, but to test the quality of faster but sub-optimal methods. 

Two heuristic approaches 

There are several ways to do a faster but sub-optimal contour 
generation. Algorithms of that kind can be subdivided into two 
classes. The first class includes algorithms which, starting at an 
arbitrarily selected initial vertex, extend lines until the accuracy of the 
line approximation is no longer satisfactory [8],[10],[11]. We will 
name this kind of algorithm £ontinued-Line-Extension(CLE)- 
algorithm. A basic algorithm can be implemented with quadratically 
increasing computation costs (see figure 2) though faster realizations 
are known[8]. The initial vertex v mrt in that algorithm is the vertex 
with maximum jc- value since a vertex at a local maximum of the 
boundary is a likely candidate for an approximation. 

begin 

V a = (^start>' 
v actual~ v start+l> 

while v actual not equal v start + n do 
begin 

if ciev (start, actual) > maxdev then 
begin 

V c - V c" (Vactual-1>>- 

V = v 

Y start v max> 

(v max being vertex with maximum deviation,) 
end; 
else 
begin 

^actual = v actual+l> 

end; 
end; 
end. 

figure 2: Contour approximation by continued line extension. 

Another class of algorithms uses a different approach. It starts with 
two vertices {v C (i)y C (2$ which form two initial line segments 
<v C (]),v C (2)> and <v C (2;> v c(7) x Ttey ^ subject to an iterative 
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subdivision process where those line segments in V c are subdivided 
which do not satisfy the accuracy criterion. The subdivision takes 
place at the vertex of maximum deviation from the segment. An 
algorithm of that kind was suggested by [6]. 

begin 

V c=< v c(l)' v c(2)> m > 
subdivide = on; 

while subdivide is on do 

begin 

subdivide = off; 

for each segment <v c(i)f v c(i+i) > ^° 
begin 

if subdiv (<v c(i) ,v c(i+1 )>) is neccessary do 
begin 

find V div of max. distance to <y C (i)^c(i+i) > ' 

Vc = Vc U {v div }; 

if dev(c(i) fdiv) > maxdev then 

begin 

subdiv (<c (i) f div>) = necessary; 
subdivide = on; 
end; 

if dev(div,c (i+1) > maxdev then 
begin 

subdiv (<div,c (i+1) >) = necessary; 
subdivide = on; 
end; 
end; 

for each triple (c(i-l) ,c(i) ,c(i+l) ) do 
begin 

if dev (c (i-1) , c (i+1) ) < maxdev then 
begin 

end; 
end; 
end; 
end; 
end. 

figure 3: Contour approximation by subsequent subdivision. 

After subdivision adjacent segments may be merged to a single 
segment, if that segment still meets the accuracy requirements[5]. We 
name this algorithm Subsequent-SubEivision(SSD)-algorithm. The 
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algorithm also exhibits quadraticly increasing computation costs (see 
figure 3). 

The two initial vertices of the SSD-algorithm are those which form 
a line with minimal angular deviation from the first principal axis of 
all vertices, (The first principal axis <fpa> of a set of vertices is an 
axis with minimal mean square distance to all vertices[3].) However, 
to exclude initial lines which are too short we decided to add a 
weighting factor which is proportional to the length of a line segment: 

priority(<v it Vj>) = length( <v it vj>) I angle( <v if Vj>,<fpa>) 

The vertices of the line with the highest priority score are choosen 
to be the initial vertices of V c . 

Results 

The two heuristic approaches were compared with the optimal 
contour approximation method by testing the algorithms on binary 
scenes of PET scans of the head. The accuracy was set such that all 
approaches reduced the number of boundary vertices by the order of a 
magnitude. The optimal method, however, resulted in 30% to 80% 
less vertices than the two heuristic approaches. Of the heuristic 
methods the SSD-algorithm had a better performance than the CLE- 
algorithm. On average the SSD-algorithm needed about 30% fewer 
vertices than the OLE-algorithm. The reason for this is perhaps the 
possibility of the SSD-algorithm to alter the contour after subdivision 
by joining adjacent line segments whereas the CLE-algorithm does 
not allow the removal of a vertex once it is selected for the contour 
approximation. 

Conclusions 

In this paper we have demonstrated a new algorithm for an optimal 
contour approximation. We used the method to test two heuristic 
approaches. We found that the method of subsequent subdivision was 
superior to the method of continued line extension though both 
algorithms did not achieve optimality. 
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Abstract 

Segmentation of meaningful anatomical objects from raw intensity data 
is a fundamentalal operation in computer assisted analysis of multidi- 
mensional medical images. Due to high demands for the reliability of 
the segmentation fully automatic segmentation procedures applicable to 
MRI images are unlikely to be accepted in a clinical practice. We present 
an implementation of a segmentation concept which is based on 2D and 
3D edge detection algorithms. The edge elements which have been detec- 
ted automatically are complemented by efficient interactive tools. These 
enable the user to construct the final objects while incorporating his ex- 
pert knowledge in the iterpretation. The segmentation tools combined 
with efficient techniques for visualization and quantitative analysis cons- 
titute the basic modules in an image processing workstation for clinical 
applications and utilizing volume data. 

1. Introduction 

Recent advances in magnetic resonance imaging (MRI) with respect to 
image quality and data acquisition speed have brought the use of MRI 
volume data into routine clinical use. In addition to the development of 
data acquisition also efficient tools for the analysis and visualization of 
volume data are required for an optimal exploitation of the information. 
During the last few years a number of 3D visualization techniques have 
been developed. They have demonstrated their usefulness in various 
medical applications such as in diagnosis and in planning and monitoring 
of therapies [1], 

The successful development and implementation of fast and efficient tools 
for volume visualization should not obscure the fact that automating the 
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segmentation of MRI volume data has remained a largely unsolved prob- 
lem. Promising results for MRI data have been obtained by extending 
standard edge detection algorithms into 3D and applying them to volume 
data [2-3] or through the use of multispectral classification techniques [4]. 
However, in practice the user frequently has to draw the object contours 
manually on each slice of the data volume to obtain a clinically valid and 
accurate result. 

Segmentation of meaningful objects from raw volume data is an essential 
and integral step towards computer assisted analysis and efficient visua- 
lization of multidimensional medical data. Due to the high demands for 
accuracy and reliability of the segmentation in medical applications fully 
automatic procedures are unlikely to be accepted in a clinical practice. 
Thus successful practical segmentation techniques must be able to co- 
mbine automatic algorithms with efficient and user-friendly interactive 
methods. 

In this study a practical segmentation scheme based on standard edge 
detection algorithms is presented. Edge elements which have been deter- 
mined automatically are interactively modified and grouped into closed 
object contours. This interactive stage is based on minimizing the energy 
of a spline curve by taking into account both external constraint forces 
and image forces [5]. The external constraints are imposed by the user 
to guide the contour to take a desired route and the image forces are 
tuned to pull the contour towards specific image features such as edges 
or lines. 

The development of practical segmentation techniques for MRI images 
forms part of a research effort which aims at developing a workstation 
for 3D medical image processing workstation based on standard com- 
puter hardware. The basic software components of the system consis 
of segmentation, visualization and analysis techniques for volume data 
together with the necessary administrative tasks. The software will be 
tailored to suit specific clinical fields by application specific modules. 

2. Segmentation of multidimensional data 
2.1 Segmentation scheme 

Medical data typically represents an image interpretation task for which 
the correct interpretation requires very high level expert knowledge. The 
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interpretation may even differ depending on the actual application of the 
results. Segmentation which is only based on low level techniques clearly 
cannot lead to a reliable result. 

The central feature of the segmentation scheme developed in this study is 
that in addition to any automatic procedures one has to provide the user 
with efficient interactive tools to manipulate the output of automatic 
algorithms. These tools allow the user to direct the segmentation result 
toward the desired solution. 

2.2 Edge detection 

The interactive tools have to be supported by automatic procedures 
which provide a starting point for the refinement of the result. Edge 
detection has been chosen to form the basis of the initial automatic stage 
of the segmentation scheme. 

For edge detection, we currently use either the Canny operator [6] or one 
of two approximations of the Marr-Hildreth operator [7] (Difference of 
Gaussians or Laplace of Gaussian). All the currently used edge detection 
methods have been implemented both as 2D and 3D operators. During 
the segmentation procedure volume data can be considered as a set of 
2D slices or a true 3D volume. The 2D slices for the segmentation can 
naturally be selected orthogonal to any of the three main axes of the 
volume. For a 3D application of the operators the data has to pre- 
interpolated into cubic voxels. 

A control over the desired image details is provided by the half width a of 
the Gaussian function which is contained in the operators. The Difference 
of Gaussians operator contains two Gaussian functions; the ratio of the 
width values between the two Gaussians remains constant. For large 
values of <r noise is efficiently smoothed out, but also important details 
and accuracy in the edge locations may be lost whereas the use of small a 
results in good reproduction of details but also in an increased noise level. 
The actual width a to be used is currently determined experimentally 
for each case separately. 

Besides the resolution level the user can control the output of the edge 
detection by a selection based on the edge strength. This is carried out 
interactively on the screen by modifying the threshold operating on the 
output of the edge detection operator. At this stage the user also selects 
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those edge elements which are to build up a basis for the successive 
refinement of the object contours. 

2.3 Interactive object refinement 

The automatic edge detection procedure is complemented by techniques 
relying on interactive operation to obtain a reliable segmentation result 
with a minumum of user interaction. The user can pick up edge elements 
from different resolution levels and combine edge elements representing 
different levels of edge strength to form initial components of object 
contours. These initial components can be declared fixed or they can be 
allowed variable subject to similar procedures that are applied on the 
edge linking into closed object contours. The object refinement stage 
can even be carried out completely from scratch without a prior edge 
detection step. 

The edge elements are modified and linked together using an approach 
based on minimizing en energy function [5]. An energy function is deter- 
mined for a line between two points consisting of image, line, and external 
forces and a local minimum for this function is determined iteratively. 
The process is initiated at a straight line between the two points or a 
result from the edge detection process. 

Image forces which are contained in the energy function currently can 
be tuned to follow edges, ridges, or valleys in the image. The line forces 
tend to smoothen the line avoiding sharp comes. Through the inser- 
tion of external forces the user can additionally pull the line towards a 
selected location or push it away from another one thus directing the 
energy function from one local minimum to another desired one. The 
relative weights for the components of the total energy function can be 
modified interactively thus providing the user with powerful tools for 
semi-automatic segmentation. 

2.4 Postprocessing 

All the 3D visualization techniques that we currently are using are based 
on direct surface reconstruction from voxels. The image data is stored as 
an array of voxels throughout. The segmentation results occupy either 
predefined bit planes of each voxel or build a voxel array of their own. In 
order to obtain classified voxels the object contours on each slice are filled 
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using standard graphics algorithms followed by a 3D connected compo- 
nent labelling procedure [3] to finally yield the 3D object description in 
the voxel format. 

3, Implementation 

The segmentation algorithms and the user interface have been developed 
using Symbolics workstations and Common Lisp as the programming 
language. The algorithms form an integral part of a menu-driven image 
processing software package developed at Technical Research Centre of 
Finland, Graphic Arts Laboratory. The main applications of the package 
are multidimensional medical images and binary images such as maps and 
technical documents. 

The software is currently being integrated into the UNIX operating sys- 
tem using the C-programming language under the X- Windows environ- 
ment as part of a software development project aiming at a workstation 
for multidimensional medical image analysis. All visualization algori- 
thms are voxel based utilizing the so-called prebufFer algorithm [3] for 
fast surface reconstruction. 

4. Conclusion 

A segmentation scheme for medical volume data has been presented. The 
scheme incorporates low level image processing techniques in the form of 
edge detection filters. The edge detection result is complemented with 
high level expert knowledge through an interactive procedure using the 
automatically detected edges as input to refine the final object contours. 
The segmentation techniques have been integrated in in a voxel based 
3D visualization package. 
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Medical images, like images from oilier domains such as manufactured objects 
and natural scenes, are full of structures, groups of features with some form 
of spatial regularity. In many domains parallel edged structures are of partic- 
ular interest ami (he automat ie extraction of those structures has been ad- 
dressed by many researchers. 

For instance, there has been considerable work in identifying arteries by 
tracking where an operator specifies a location in the image thai is inside an 
artery and an initial tracking direction (1,2), or by an analysis of small areas 
of the image which may indicate the presence of an artery. This and other ev- 
idence is combined to produce possible artery traces {.\4). These processes are 
invariably domain dependent, looking only for arteries, with some form of 
encapsulated domain knowledge which makes them inappropriate for other 
tasks. This is essentially a local Pattern Recognition approach. 

However, if a more global analysis is performed using ideas developed from 
Computer I'ision, then it is possible that processes can be developed which will 
identify features from a wide range of domains with no operator intervention. 
The concept of interest is that of a Semantic Pre- cur sot if t which a structure 
is identified before it is named as a particular object. In this paper parallel 
edged structures are of particular interest. 

There are many types of medical image in which parallel edged structures ap- 
pear. One obvious case is an X-ray angiogram, but there are other modalities 
in which a parallel edged structure is a useful landmark in locating the feature 
of interest. This paper describes a set ol processes which will identify parallel 
structures over a wide range cif modalities, three f*f wlttcli are shown in the re- 
sults. In order to emphasise the semantic free nature of (lie identified struct lire 
they are called Ribbons, The processes are edged based, starting with a local 
assessment of the presence of an edge and building up larger structures until 
all the ribbon structures in the image have been identified. A ribbon niotJel iiids 
in the resolution of ambiguous data. A full description of the processes ean be 
found in (7). 
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The Ribbon Finding Processes 

Edge Detection 

The First process is to identify edge points that occur in the image. At this stage 
no attempt is made to assess their usefulness as edges of ribbons. Hie CANNY 
operator (6) has hecn found to give good quality edge data in a number of dif- 
ferent domains since it gives a good signal to noise ratio at an edge. The output 
of the process is a list of points that have been identified as edge points. Each 
item in the list has the X and Y coordinate of the edge point to sub-pixel ac- 
curacy, the orientation of the edge point from 0° to 360° and the strength of 
the edge point which is a measure of the contrast across the edge. The edge 
points arc thresholded to eliminate those that are below a certain strength and 
thus likely to be of less interest than points above the threshold. 

'Hie next process is to group the edge points into larger entities. Since we per- 
ceive adjacent edge points in the image as connected and a set of connected 
edge points as one perceptual group, an edge, the larger entities will be an edge 
list. Each point in an edge list is classified as a terminator (the end of an edge 
list), a line, point (a point inside an edge list) or a junction (coincident termina- 
tors from two or more edge lists). 



Finding all Ribbons 

Having identified continuity along the edge segment and grouping the relevant 
edge points into edge lists, we are now in a position to look for ribbons. A rib- 
bon is defined by two edge lists in which there arc points which satisfy distance 
and orientation constraints. One edge point is part of a ribbon edge if there 
is a point in a second edge that is close, within the constraints. The corre- 
sponding points define the Line Generators of the ribbon and an ordered list 
of Line Generators defines a ribbon. 



Self-parallel edges arc single edges which 'double back' on themselves to 
produce close, parallel, edge points of opposite orientation. I bis type of edge 
is surprisingly common, and occurs most often when there is good contrast 
between the ribbon and the surrounding areas of the image. 
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Figure 1. Edges generate ribbons: Ivdgcs 1:1, V.2 and Fi3 gencralc ribbons RI, 
R2 and R3. "I'fic dotted lines arc the I .if ic Generators and tbc 
dashed lines are the ribbon centre lines. 



Removing the Ambiguities 

Since the search for ribbons is exhaustive it is possible for two ribbons to be 
identified which use the same part of tbc same edge. The use of tbc edge is said 
to be ambiguous. There arc two broad categories of ambiguity, as shown in 
Figure 2. 

Ambiguities arc resolved by deleting data. Wide Ribbon Deletion is used to 
delete a whole ribbon. If the ambiguity is of the type (A) in Figure 2, deleting 
the wider ribbon R2 will remove the ambiguity. Common Point Deletion is used 
to 'erode' the ends of ribbons by deleting one Line Generator at a time until 
the ambiguity is removed. In Figure 2 (B), the deletion of the individual Line 
Generators at the point of overlap will remove the ambiguity and still retain 
most of the data in Rl and R2. 



Finding the Junctions 

After the ambiguities between ribbons have been removed, the remaining rib- 
bons will consist of a number of disconnected ribbon pieces. IF the ends of the 
ribbons are close (to some criterion) then, in some domains, it may be appro- 
priate to try to join the broken ribbons. For instance, in cardiac angiograms 
it may be reasonable to make an initial assumption that the ribbon structures 
will be continuous and an attempt to repair the breaks in some way is accept- 
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Figure 2. The Categories of Ambiguity: 

(A) Wide ribbons. The ambiguity is resolved by completely deleting 
the wider ribbon, R2. 

(H) Overlapping ribbons. The ambiguity is resolved by deleting the 
I /me Generators causing the ambiguity. 



able. T here arc domains in which it is important to identify the more complex 
structures where three or more ribbons meet, a junction. For instance, in 
angiograms, the point at which an artery divides (a bifurcation) is of consider- 
able interest and the bifurcation can appear in the image as a junction of three 
ribbons. The processes repair the local breaks and identify the junctions, using 
evidence existing in the image and already extracted by an earlier process. 

One piece of evidence is the proximity of the ends of the ribbons, and the other 
is continuity of an edge between two ribbons. If continuity is combined with 
proximity, the inference of a ribbon join or junction can be made. 



Results 

The processes described above were run over three types of medical images. 
Figure 3 is a digitised radiograph of one of the views of one stage of the heart 
beat cycle. It shows the cardiac arteries after the injection of the radio opaque 
material. Figure 4 is an IMRI image and is a horizontal slice located just above 
the cars. The structure of interest is the bright ribbon of the scalp. Figure 5 
is of a preparation, viewed through a microscope, of a cross-section of muscle 
fibre. The structures of interest arc the bright ribbons between the darker 
fibres. All the images arc 256 pixels square. 
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There is not sufficient space to show the results of the processes at each stage. 
However, the figures below show a representation of the extracted ribbon data, 
after ribbon extension and junction identification. 

The only tuning of the processes is by two sets of global parameters, one to 
control the thresholding of the edge points, the other to control the dimensions 
of the structure being investigated. 

In general, where a parallel structure exists in the image, a ribbon has been 
identified. Inspection of the edge lists produced by the edge detector shows that 
where there is no ribbon, there is insufficient edge data to infer a ribbon. The 
parallel structures arc incomplete in some cases. Tliis is to be expected - no 
process will produce perfect results in the type of images being processed here. 
In some cases, parallel structures have been identified which arc not caused by 
the features of interest in the image. This is also to be expected. The processes 
arc identifying 'semantic prc-cursor' structures which will require a later 
understanding process to separate out the 'real' objects from the 'artefacts'. 
What is especially interesting is that very few artefacts have been identified. 

lite results of the intermediate stages are described in the full report on this 
work (7), together with performance figures and implementation details. 



Conclusions 

The results show that the methods described here can produce good quality 
descriptions of parallel edged structures in images from a number of different 
domains. The processes arc identifying 'semantic precursors' of objects, such 
as arteries in the HEART image, the scalp in the HEAD image and the 
boundaries between the fibres in the MUSCLE image. The processes have no 
knowledge of arteries, heads or muscle fibre. 

The accumulation of evidence for the existence of a ribbon and the use of a 
ribbon model (defining how a ribbon can appear in an image), results in the 
ribbon structures being accurately identified and located. 
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Figure I IK ART: I lie original image is an X ray cardiac angiogram aflcr 
the injection of radio opaque dye. The identified ribbons arc shown 
on the right. 




Figirre 4. MEAD: The original image is an MR1 image. "1 "lie i den I tiled rib- 
bons are shown on the right, 
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Figure 5. MUSCLE: The original image is a cross -section of muscle fibre. 
The identified ribbons arc shown on the right. 
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The rapid evolution of computer-based technologies in diagnostic radiology is 
creating a digital environment that is bringing unprecedented power to radiologists. 
Clinical information systems and image management and communication systems 
can deliver a wealth of clinical and image information. New systems are being de- 
veloped that focus medical knowledge on this clinical and image information. This 
added dimension of automatic access to highly specific knowledge offers the poten- 
tial for radiologists to exhibit expertise far beyond currently recognized human limita- 
tions. 

Decision making in diagnostic radiology occurs at many levels. Our primary focus is 
on diagnosis. To make a diagnosis, we must understand the implications of the 
patient's clinical history. We often must help the patient's physician choose the most 
appropriate imaging examination or examinations and then make sure that the studies 
that are performed maximize information yield. Our greatest intellectual challenge 
begins when the images are before us. What abnormalities can we identify and what 
do they mean in the clinical context in which they are presented to us? Have we seen 
all there is to see and have we accurately characterized what we have seen? Have we 
not only understood the significance of each individual finding, but have we put it all 
together correctly? Ideally, our analysis of these clinical and radiographic findings 
lead to a specific diagnosis. Often, however, we must settle for a list of possible 
diagnoses. Then we must progressively narrow this differential diagnosis list - first 
by applying knowledge or "expertise** and then by recommending additional imaging 
and sometimes clinical workup. 

How can computers help us make decisions in diagnostic radiology? 
Computers can either make decisions directly or can act as a 'knowledge servant" 
that helps us make decisions more effectively. For nearly two decades, investigators 
have been interested in computer diagnosis 1 . Computer diagnosis systems often 
directly process clinical data and attempt to make a diagnosis by simulating human 
thought processes. This may be effective in certain clinical settings where diagnoses 
can be reduced to well defined algorithms and where clear cut, often quantitative, 
data are available. Despite some impressive successes in the laboratory 2 , few of these 
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systems are used clinically. There have been several interesting systems that have 
been built in diagnostic radiology that deal with specific problems such as bone 
tumor diagnosis, and the evaluation of the arthritides 3 * 4 * 5 . 

Decisions in radiology often do not lend themselves to this approach, however, 
because of the often very subjective nature of image interpretation. As a result, the 
computer may be more effective if it can support our own thought processes, rather 
than replace them. One way to do this is to cast the computer in the role of a *knowl- 
edge servant' whose job it is to present us with knowledge that can enable us to make 
decisions more effectively. This would provide a very complementary relationship 
between us and computers. Humans, after all, have poor memories, and process 
information slowly (approximately 50 bits per second). We also only think about one 
thing at a time (we are serial processors). We excel, however, at making intuitive 
judgements about things that we do remember. Computers, on the other hand, have 
almost unlimited memories and can process and retrieve information at very high 
speed. It is much more difficult to get computers to make the kind of intuitive 
judgements that we make easily. Therefore, it makes good sence to combine these 
different human and computer strengths so that the combination is more powerful 
than either separately. One of the ways that computers can serve us in medicine is to 
filter information for us. Since we process information so slowly, we can't possibly 
keep up with all of the important information that is available. Information from the 
patient's clinical record, books, journals - we are overwhelmed. It is daunting to 
realize that new knowledge is created so rapidly in medicine, that the entire body of 
medical knowledge doubles every five years. The result is that it is becoming almost 
impossible to practice state-of-the-art radiology, even if we only practice a subspe- 
cialty. Fortunately, the computer can help us stem this flood of knowledge by 
filtering it for us. It can evaluate the clinical data that is known about our patients as 
well as the observations that we extract from their images and delivery carefully 
selected information to us in a way that helps us understand their implications. 

We have been exploring many of these issues at Yale through the Image/Icon family 
of prototype decision support systems 6,7 . These artificial intelligence-based systems 
seek to 'understand' the issues the radiologist is dealing with by directly evaluating 
his or her case description together with the patient's clinical history. These systems 
present selected or 'filtered' information based on these findings in two different 
ways. First, the system produces an English prose discussion of the significance of 
the issues raised by the case. Second, the system retrieves and displays images from 
an image database that are likely to help strengthen the radiologists working diagno- 
sis, or cause him or her to consider another diagnosis. Several systems have been de- 
veloped to bring computer-based advice to bear on the process of selecting diagnostic 
imaging procedures 8 * 9,10 . 

Direct computer interaction with images may play an increasingly important deci- 
sion support role in the future. This may involve standard image processing methods 
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to improve contrast resolution, manipulate spatial frequency, energy subtraction, etc. 
In special circumstances, some computer systems have begun to be able to extract 
'information' directly from images. Interesting research has been done in computer 
identification of lung nodules 11 , breast masses 12 , breast calcifications 13 , and in the 
automatic identification of such anatomic structures as cardiac chambers 14 . It is 
important to emphasize that widespread automatic image interpretation by computer 
is not likely to occur in the foreseeable future. 

The ImteHiefflt Radiologic Workstation 

Image management systems, clinical data management systems and knowledge-based 
decision support systems have been evolving separately. The sophistication of the 
digital environment that is becoming commonplace in diagnostic radiology offers the 
opportunity to bring these separate technologies together in the form of an intelligent 
radiologic workstation. This union will produce a device of unprecedented power that 
will not only display and process images, but will provide a portal to vast libraries of 
information. It will filter this information for us and focus it on our immediate 
clinical problems. It will do this in real time - at the moment of truth - when we are 
making clinical decisions. The result will be a tool that will significantly extend our 
intellectual powers. 
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Abstract 

This paper discusses the applications of 
informatics to Radiology. Informatics is the 
study or use of information and system 
sciences in association with computers (1). 
In this paper, I will discuss the 
radiographic decision making process, how 
this can be modeled with advice programs , 
difficulties in writing and verifying an 
advice program, and potential benefits of 
this research. 

Introduction 

Radiology is an excellent area for applications 
of Artificial Intelligence (AI) research because 
much of the reasoning process of radiologists is 
rule-based and many AI programs use rule based 
programming systems. Radiologists will 
frequently use rules such as: 

If the heart is enlarged, and 

if there are Kerly B lines, and 
if there is alveolar edema, 
Then, there is a high likelihood that the 

patient has congestive heart failure. 

It is felt that this research will lead to 
improving the rules that radiologists use to 
link radiographic features on a radiograph to a 
diagnosis, develop computerized representation 
of radiographic knowledge, help standardize 
radiographic terminology, ease report generation 
from a radiographic diagnostic workstation, 
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improve the user interface of a diagnostic 
radiology workstation, assist with teaching 
radiographic concepts to radiologists and 
testing performance of future radiologists. 

Radiographic Decision Making Model 

The principle task of the radiologist is to 
analyze images and provide a diagnosis. Through 
training, the radiologist has learned to examine 
an x-ray and integrate the shadows present on 
the image to give a diagnosis. If asked to 
explain a diagnosis, the radiologist will 
describe the radiographic features that are 
present on the image and how he reasoned about 
these features to give the diagnosis* He will 
also take into account clinical details of the 
case . 

From the above discussion, it follows that 
the radiologic decision making process can be 
modeled by a two step process. In the first 
step, the radiologist analyzes the images and 
either implicitly or explicitly defines a set of 
features. The next step is to reason about the 
presence and absence of the features on the film 
and incorporate these into a diagnosis. I have 
chosen to use the term radiographic features in 
this discussion. This process was first 
described by Blesser and Ozonoff in 1972 (2). 
They described a three-step model, but the first 
step consisted of image formation. Others have 
discussed the distinction between features and 
reasoning (3-5). 

The most common reaction to a discussion of 
AI , radiographic features, rules, etc., by a 
radiologists is: "I don't use rules, features, 
or any of that non-sense. I look at a film and 
I ^now* the diagnosis." However, this misses 
the point! Yes, a trained radiologist "just 
knows" the answer, but during training, he or 
she learns what the radiographic features are 
and how to reason about them. It is a sign of 
being a trained radiologist that this two-step 
process has been transformed into a one-step 
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process. Psychologists call this process 
chunking . 

One categorization of artificial intelligence 
programs in Radiology is that there are programs 
that attempt to categorize the "signal", i.e., 
the image data, and whose output is a list of 
radiographic features. The second category of 
program uses this set of features as input and 
produces advice (such as a diagnosis) as output. 
A radiologist performs both of these functions. 
I have chosen the terms front-end program (FEP) 
and advice program (AP) respectively to describe 
these two functions. Advice programs could be 
expert systems, neural nets, or statistical 
programs. These can be submodules of a larger 
program. 

When building a machine that is capable of 
radiographic image interpretation, it will have 
to "learn" by being programed with an internal 
representation of these same radiographic 
features, rules. At the end of the development 
process, a system that has FEP and AP 
capabilities should, likewise, be able to 
"examine" the image and "just know" what 
diagnoses may be present; that rules, frames, 
confidence factors, fuzzy logic, etc., may have 
been used in the process is immaterial. 

It has been shown that perception of 
radiographic features has significant inter- and 
intra-observer variability (6-9) # An 
implication of this is that it is difficult to 
validate the accuracy of an advice program. To 
obtain a set of cases which has "known" findings 
on the images and proven pathology is difficult 
(10-11). This is a problem only when one tries 
to identify the radiographic features present on 
each study. For instance, when the system under 
observation consists of both a FEP and an AP 
(whether this system is human or computerized) , 
then the output of the system is a diagnosis 
which can be compared with some external 
standard of "truth". When trying to identify 
the radiographic features of a set of cases 
(i.e., when trying to analyze the data that is 
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exchanged between a FEP and an AP) , there is no 
objective way to determine which features are 
present because these are the perceptions of 
individuals. A method has been proposed for 
deriving a test case features database by having 
multiple observers interpret the test cases and 
deriving a set of "consensus" features (12). 

Another problem with identification of 
radiographic features is that there are subtle 
differences in the terminology used by 
radiologists. This leads to variability in 
describing radiographic features. The havoc 
that this can cause when using advice programs 
is described by Piraino, et.al., (13) and 
Bramble (14) (Garbage In, Garbage Out). 

Constructing an Advice System 

When one attempts to construct an advice 
system, there are several prerequisites. The 
first is the availability of an adequate number 
of cases with known diagnoses and with a 
reasonable number of difficult and subtle cases 
of the diagnosis between which one is attempting 
to differentiate. 

After acquiring a suitable number of cases, 
the next step is to determine which radiographic 
features are necessary components of the domain 
and to determine which programing technique will 
be used to form the advice part of the system. 
This step can be performed by rule-based systems 
in which case the rules will have to be 
determined. Either a textbook or an expert can 
be used as the knowledge source. Traditionally, 
interrogation of experts has been used for the 
knowledge acquisition process. With neural nets 
and statistical programs, the rules do not have 
to be developed because the relationships are 
derived numerically. 

After having developed the advice program, 
the expert system should be validated. In order 
to validate computer programs, known input must 
be provided to see if the system will produce 
the correct output. I have already described 
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the problems inherent in determining the 
"correct" radiographic findings on a set of 
radiographs. After developing the test case 
feature database, the expert system can be 
validated and if necessary the rule base can be 
refined until performance is satisfactory. 

Uses of Advice Systems in Radiology 

At this point let us assume that one has 
collected an adequate sample of test cases. A 
database of the correct features on these cases 
has been assembled by consensus readings, and an 
advice program has been developed and validated 
by using the test case feature database. I am 
proposing several uses of advice programs in 
radiographic research . 

The effort of developing the test case 
feature database will be of value in and of 
itself. The inter- and intra-observer 
variability will be defined for this group of 
cases and for this diagnostic domain. If there 
is a particular feature which is perceived with 
very little agreement among radiologists, then 
the use of this feature in making diagnoses 
should be questioned. Conversely, those 
features agreed upon with a high frequency can 
be used with high confidence. The effect of a 
systematic search on diagnostic accuracy can 
also be evaluated. 

If the advice system is developed using 
expert systems methodology, then the refined 
rules can be reverse translated into English and 
used as an improved codeif ication for the 
interpretation of this category of examinations. 
This process of using rule-based expert systems 
to improve the rules used by experts has been 
called knowledge refinement (Michie, D. Personal 
Communication) . 

The problems of non-standardized radiographic 
terminology have been discussed previously in 
this paper. The use of computerized advice 
programs has pointed out a problem with noise 
which is inherent in the English language 



663 



descriptions that humans deal with fairly 
easily. However, in this decade of increasing 
computerization, standardization of terminology 
should be performed. Or if standardization 
cannot be implemented, then thesauruses of 
equivalent terms should be developed for human 
and computer use. (This might come to pass from 
the Unified Medical Language Project of the 
National Library of Medicine.) With 
computerzied advice programs, definitions of 
features including illustrations should be 
available on-line . 

Advice programs can be used to help teach and 
test radiologists. For instance, the benefits 
of various teaching methodologies may be 
compared in the following fashion: A group of 
naive students is shown a set of cases and are 
asked to interpret the cases by describing the 
findings which are present on each film and the 
diagnosis. This initial group is sub-divided 
and each sub-group is given alternative methods 
of instruction of how to interpret this category 
of exams* Subsequently, a second set of test 
cases is shown to the sub-groups. The ability 
of the individual groups both to perceive the 
radiographic findings and to make the correct 
interpretation following the different means of 
instruction can be objectively assessed in this 
manner using advice program technology. 

These systems can be used to assess the skill 
of practitioners either during initial 
certification or re-certification. The 
candidate can interact with the advice program 
by describing the radiographic features present 
and give his/her diagnosis. Perception of the 
individual features can then be compared with 
the test case feature database to see if they 
match the performance of a suitable percentage 
of peers. In addition, ability to make the 
correct diagnosis will be recorded by the system 
as well. Therefore, one can compare both the 
perception of individual findings and the 
differential diagnosis. This may ease the 
burden of oral and/or re-certification 
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examinations which the profession has to 
confront in the future. 
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Description 

The E.T.Nct (Educational Technology Network) is a one year old, free, easy to use, 
24 hour accessible, online computer conferencing system organized by the National Library of 
Medicine which is accessible via a local phone call to the Telenet (Sprintnet) computer network. 
It is dedicated to helping bring together, in a single continuously running electronic forum, 
developers and users of computer applications in health sciences education and radiology. 

E.T. Net's advanced computer conferencing software allows individuals who are 
geographically and temporally separated to unite electronically in order to participate, at any 
convenient time for them, in interactive discussions on applications of computers in health 
sciences education and radiology. Computer conferencing software is a still relatively unknown 
form of computer software that facilitates a structured and organized exchange of online 
information and ideas, and is a distinct improvement over the more widely known computer 
bulletin board software that facilitates an unstructured and unorganized exchange of online 
information and ideas. A computer conferencing system such as E.T.Nct provides the valuable 
service of allowing health sciences professionals to conveniently remain in regular contact with 
colleagues on issues when they are not physically together at face to face conferences. 

E.T, Net is organized hierarchically, and consists of several computer conferences, 
each encompassing a broad but related subject area of potential discussion. Each conference is 
further subdivided into items, with each item serving as a starting point for a discussion on a 
topic. Discussion is carried out by participants adding responses directly to the appropriate 
item. Each conference has an index of all the items within it. Each time one logs onto E.T. 
Net, for each conference they belong to, they arc shown the new responses to old items and the 
new items lit at have been added in the interval, and they arc given the opportunity to add their 
own responses to preexisting items and to add their own new items. 

E.T. Net was conceived to facilitate communications between health sciences 
professionals utilizing computers in health sciences education and Uierefore the majority of the 
conferences *ind hence discussions on E.T. Net deal with applications of computers in health 
sciences education and include: integration of computers into health sciences educational 
programs in the basic and clinical sciences, development of new and evaluations of existing 
health sciences educational software, uses of videodisc and CD-ROM hardware technologies in 
health sciences education, uses of hypermedia and artificial intelligence software technologies in 
health sciences education, computing in health sciences libraries, training and careers in medical 
informatics, and the ultimate goal of easily organizing and disseminating medical information. 

In addition, there is a conference on E.T.Nct, entitled "digitaMmages," dedicated to 
applications of computers in radiologic practice and education. Topics of discussion include but 
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are not limited to: radiologic education via hypermedia and computer aided instruction systems, 
digital radiography and picture archiving and communication systems (PACS), image processing 
and three dimensional image reconstruction techniques, teleradiology, radiology information 
systems (RIS), uses of artificial intelligence in radiology, and how computers can aid 
radiological research. 

E.T.Net uses the Caucus computer conferencing software (1) running on a 80386 based 
microcomputer using the Xenix operating system. This microcomputer is located in the 
National Library of Medicine's Lister Hill National Center for Biomedical Communications in 
Bethesda, Maryland and is directly connected to the Telenet packet switching network, A user 
connects to E.T .Net by placing a phone call to their nearest Telenet node using a personal 
computer and a modem and the Telenet node then connects them to E.T.NeL E.T.Net is 
currently configured to allow its use by up to four people simultaneously, and if necessary this 
may be expanded to eight simultaneous users in the future. As part of its mission of 
disseminating medical information to health science professionals, the National Library of 
Medicine covers the cost of running E.T. Net as well as the Telenet charges of E.T.Net users. 

A startup information packet is available directly from the Educational Technology 
Branch of the Lister Hill National Center for Biomedical Communications at the National 
Library of Medicine (2) or alternatively, one may follow the instructions in the Appendix below 
to logon to E.T.Net for the first time. All one needs to access E.T. Net is a personal computer 
with communications software and a modem. 

Participation on E.T.Net is open to all health sciences professionals and students. 
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Appendix: How to loin E.T.Net: 

1 . All you need to do to join is to dial into your local Telenet node via a local phone call. To 

find the Telenet node nearest you, dial 1 -800-835-3638 and tell them where you live 
and what speed your modem is. They will tell you the correct number to call. 
Make sure your modem is set to 7 data bits, 1 stop bit, and even parity 

2. Dial your local Telenet node telephone number. 

3. When Telenet asks for TERMINAL^, type, VV and hit <RETURN> 

4. When Telenet gives you an '<§>' prompt, type 'C 301 565* and hit <RETURN> 

5. E.T.Net will then ask you to 'Please enter network login: 1 , and you should type 

'etnet 1 <RETURN> 

6. Follow E.T.Net's prompts, and when it asks you to enter your last name, type 'newuser' 

<RETURN> Please note to be sure to type 'newuser' in lower case. 

7. Now E.T.Net will take you through a registration process. You are on your way! 

8. When you have completed the registration process, E.T.Net will ask you to "JOIN which 

conference?" Type 'users_guide* now, and E.T .Net will join you to the 'users_guide' 
conference which will teach you the basic commands you need to know to use 
E.T.Net. Alternatively, from an AND NEXT? prompt, you can type 'join 
users_guide' and be placed into the 'users_guide' conference. Once you are inside the 
'users_guide' conference, from an AND NEXT? prompt, type 'new* and you will be 
given an introduction on how to use E.T.Net. If you wish, you can save it and print it 
out for future reference. 

9. After finishing the , users_guide' conference, from an AND NEXT? prompt type 'join 

digital_images' to join the 'digitaHmages' conference where you will be able to 
participate in discussions relating to applications of computers in radiology. 

10. Remember that anytime you are confused by E.T.Net, just type 'HELP' in order to get 

assistance. 

11. When you are done using E.T.Net, type 'bye' in order to logoff. 
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Abstract 

TEACHiR is a computerized decision support system that assists in the 
interpretation of medical images. TEACHiR is an acronym for Thompson's 
Education And Consultation Help In Radiology. This program will assist 
health care providers in the formulation of a set of likely differential 
diagnoses for a given radiologic finding. The program uses the GAMUT 
system (a system which links radiological findings to differential diagnoses) 
developed by William LeRoy Thompson M.D., the first registrar of 
Radiologic Pathology at the Armed Forces Institute of Pathology. 

The system is menu driven and user-friendly, prompting the health care 
provider for a methodical description of the findings that are present on a 
particular study. Then, the program generates a list of differential diagnoses 
and displays representative images of each of the diagnoses along with 
descriptions of each of the diagnoses. 

We have completed one section of the TEACHiR program, regarding the 
interpretation of mammograms. We have selected to work on a system to 
assist with the interpretation of mammograms for several reasons. First, the 
mammogram is a very commonly ordered test which has great clinical 
significance. Secondly, by selecting a narrow field, we will be able to input 
all of the necessary clinical data into the TEACHiR program (as there is a 
relatively small subset of disease and non-disease entities that affect the 
breast). This clinical data will contain definitions, locations, incidence, 
prognosis, pertinent clinical data, pathology and representative images. 
Equipped with this additional information, the user can make a much more 
informed decision regarding the imaging study that he is viewing. We will 
also evaluate the ability of the software to assist in the diagnostic process by 
including a computerized survey which will offer both subjective and 
objective feedback regarding the utility of the program. 
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Significance 

TEACHiR represents an efficient program which can be used to train primary 
care physician's to recognize signs of pathology in radiographs. In addition 
to helping the health care provider in the management of clinical cases, this 
system provides a wealth of information for the student of radiology. A set 
of valuable rules regarding the interpretation of diagnostic images can be 
learned through the use of TEACHiR™. The program will provide a 
valuable complement to existing teaching hospital education resources. 
Residents who consult TEACHiR will obtain valuable information regarding 
the interpretation of medical images. 

Furthermore, as more radiology systems gain the ability to display medical 
images on remote workstations throughout a hospital, the TEACHiR 
program will become increasingly important. The clinician viewing a study 
which has not yet been interpreted by a radiologist will be able to consult the 
TEACHiR program to compare the study with the files contained in 
TEACHiR. Thus, TEACHiR will be the first available on-line help system 
in hospitals that adapt this new technology. 

All of the above information is conveyed to the physician through the use of 
a single personal computer system. The TEACHiR program represents a 
cost effective, and convenient manner of presenting this large amount of 
textual and graphic information through the use of CD-ROM storage media 
coupled with high resolution displays which are now currently available at a 
reasonable cost 

Database Construction 

Text 

In addition to providing a listing of differential diagnoses and representative 
medical images, TEACHiR will contain an extensive clinical data base. This 
data base will contain definitions, locations, incidence, prognosis, pertinent 
clinical data, pathology and representative mammograms. The database will 
consist of the gamut system as it pertains to the breast. 

In order to facilitate the entry of the extensive amount of textual material 
needed by this program, an optical character recognition program has been 
used for this task. The software, Accutext™, along with a flat-bed scanner 
and a Macintosh Ilex equipped with eight megabytes of random access 
memory has been utilized for this task. 

Images 

The image database has been be digitized and stored through the use of a 
Canon video camera attached to a Nu Vista 32 bit image capture and display 
board, and a Truevision VIDI/O video decoder box which communicates with 
a Macintosh Hex computer with eight megabytes of random access memory. 
Images have been captured with eight bits of color resolution, allowing 256 
shades of gray. 
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Software 

With the advent of Hypertext* by Theodore Nelson in 1972, the concept of 
linking information in unique ways was introduced. A commercial 
application of this idea was introduced by Apple Computer with their 
program, HyperCard. Although HyperCard is a powerful tool for the rapid 
construction of sophisticated software, it lacks the ability to display images 
that have color or gray-scale. Recently, a program called "SupeiCard" was 
introduced by Silicon Beach Software. This program is compatible with 
HyperCard; however, it allows for the use of color and gray-scale images. 
Through the use of SupeiCard, we are able to link text and graphical 
information in a very convenient manner. SupeiCard facilitates for the 
interconnection of graphics, sounds, and text to allow for flexible and creative 
multi-media computer programming. 

Hardware 

The immense memory storage requirements of the TEACHiR program will 
be met by the use of the compact disk (CD) for data storage. These CDs are 
similar to their music playing counterparts- however, when they are plugged 
into a computer, they can instantly access 500 megabytes, of information 
(2). Five hundred megabytes of information is equivalent to 250,000 single 
spaced typewritten pages, and as a further analogy, the complete text of the 
Encyclopedia Britannica consumes only 80 megabytes of text. Thus, a 
single compact disk can contain all of the information included in six 
complete sets of the Encyclopedia Britannica (2). The cost of these units 
continues to decrease and are now available in the $200 range in quantity. 
An average radiographic image that is digitized using the method described 
above takes about 100 Kilobytes of storage space. Thus, a CD-ROM holds 
680 megabytes of data is capable of storing 6,800 diagnostic images. 

The TEACHiR program will run on a wide variety of hardware. We will 
have two versions of the software - one that runs on the IBM PC and the 
other that runs on the Macintosh computer. For the Macintosh version, the 
minimum requirements for hardware will be a Macintosh II series computer 
and a CD-ROM drive. For the IBM version, TEACHiR requires a PC 
equipped with a CD-ROM drive. For either system, a hard-drive will be 
advantageous along with a high resolution color display such as a VGA 
display for the IBM or the standard eight bit color system for the Macintosh. 
Thus, TEACHiR is designed to run on commonly available hardware. 
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Evaluation of TEACHiR 
Survey 

The software will be evaluated for its utility in assisting with clinical 
decision making in both objective and subjective evaluations. For the 
subjective evaluation, the TEACHiR software asks the user, if the 
information provided was complete, helpful, clinically relevant, and presented 
in a user-friendly manner. 

Testing 

Furthermore, the TEACHiR program will be evaluated on the basis of 
testing. Fourth year medical students at The George Washington Medical 
Center will be provided unknown radiographic cases and asked to render a 
decision. Two groups will be established, one group of twenty students, will 
have access to the TEACHiR program and a second group which will not 
have access to the system. The results of the ability of the two groups to 
come up with correct diagnosis will be determined and the utility of the 
TEACHiR program will be assessed. To increase objectivity of the study, 
the images that are used for testing purposes will not be included in the 
TEACHiR database. 

Summary 

The TEACHiR project is being developed in three stages or phases, these 
phases are summarized as follows. 

In phase I, we have completed one section of the TEACHiR program, the 
section regarding the interpretation of mammograms. This section will 
include complete the construction of a gamut system for the interpretation 
of mammograms, as well as the incorporation of representative images of 
each of the possible diagnoses mentioned in these differentials. In addition, 
we will provide an on-line help directory which will assist with the diagnosis 
of each of the radiographic findings. We will also evaluate the ability of the 
software to assist in the diagnostic process by including a computerized 
survey which will offer both subjective and objective feedback regarding the 
utility of the program. 

In phase II, we will add additional areas to TEACHiR to include the most 
commonly imaged areas of the body, providing sufficient information to 
assist the primary care giver to assist in the interpretation of medical images 
that comprise the most common diagnoses that are seen by these health 
professionals. 

In phase III, we will market this product to teaching hospitals and large 
group practices. We will also sell the necessary hardware (CD-ROM player) 
to increase the potential user base of the software. 
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We do not intend for the TEACHiR program to replace the expertise of a 
radiologist in any manner. The TEACHiR program will provide several 
resources which will enhance the ability of the physician in the interpretation 
and management of clinical cases. By providing images for comparison, 
rapid access to medical text books, and providing lists of differentials for a 
given radiologic finding; the physician will have many valuable tools to help 
in the interpretation of medical images. 
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Background 

For many years, radiologists have been attempting to keep 
track of interesting cases, in order to teach medical students, 
residents, or other physicians, as well as for research purposes. The 
American College of Radiology (ACR) Index for Roentgen Diagnoses was 
developed expressly for managing such a teaching file. 1 The ACR 
Index is organized into two codes, one for anatomy and one for 
pathology. 

One of the earliest "systems" employed with the ACR Index used 
specially designed manual cards called McBee "keysort" punch cards. 2 
These cards allowed case selection by anatomy or pathology codes by 
placing a rod through the punched out tabs on the margins of the cards. 
Those cards that fell away when the stack was shaken were the cards 
selected. Obviously, large files would present a few difficulties with 
this approach. 

As soon as computers became available in Radiology 
departments, they were utilized for managing teaching files. One such 
system used mark sense cards for data collection in order to capture 
the ACR codes as cases were interpreted by the radiologist. 3 ' 4 Cases 
of interest could then be retrieved based on combinations of anatomy 
and pathology codes. The Radiology Information System (RIS) would 
then inform the user where the films were stored as well as provide 
information about the patient (including reports). However, this 
system does not capture the images or provide any structured 
teaching. 
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Computer-assisted instruction (CAI) refers to the use of 
computers to actually teach. The student can learn at his/her own 
pace and convenience. Areas of weakness receive reinforcement 
whereas areas of strength are skipped. Minimal acceptable levels of 
learning are ensured in a non-threatening environment. Perhaps the 
most important advantage of CAI relates to the variable ability of 
students to absorb information. The question and answer mode of 
teaching in CAI seems to be superior to the more passive learning with 
textbooks and lectures. 

CAI has been shown to be effective in medicine. 5 * 6 Teaching 
residents how to handle relatively uncommon contrast reactions 
without practicing on real patients is one example of a successful 
application in Radiology. 7 However, one of the greatest disadvantages 
of CAI is the difficulty and increased time for course preparation. 
Also, students and teachers need ready access to the appropriate 
computers. Since there is no immediate dialogue between teacher and 
student, some course materials are simply not well suited for this 
approach. Anything other than textual material such as pictures or 
graphics can not be handled other than with an accompanying booklet. 

Interactive video adds to CAI moving pictures and sound. 
Thousands of images can be stored on a single videodisc. 
Unfortunately, it is very costly to master the videodisc and there are 
no standards for the videodisc equipment or computer interfaces. The 
images are no better than video quality which limits the applications 
in Radiology. The ACR Institute (ACRI) recently began production of a 
series of videodiscs to make the ACR Learning File available at a 
reasonable cost. The production techniques utilized for this series 
maximize the resolution capabilities of videodisc technology and 
create quite acceptable images for many cases. 

Digital imaging has only recently been an alternative to 
videodisc, as the cost of memory and disks has fallen and as personal 
computers have become more powerful and available. New laser 
digitizers create very high-resolution images and displays provide 
overlay techniques allowing the appropriate labeling of structures on 
the image. Updating images is quite easy and sophisticated authoring 
software helps the teacher create new courses. The ability for the 
teacher or student to manipulate the image (adjust window, level, 
position) is also a major advantage over videodisc. However, moving 
pictures can consume much disk real estate and require a more 
expensive computer. As with videodiscs, there have not been any real 
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standards in either the hardware or software, creating the situation 
that requires specialized equipment for each program desired. 

DRILL Architecture 

In order to overcome the lack of standards in these digital 
imaging workstations and to build user-friendly yet powerful 
educational tools, the DRILL program was created as a multi- 
institutional project with external funding under the auspices of the 
Radiology Information Systems Consortium (RISC). The overall 
architecture for DRILL is shown in Figure 1. Conceptually a 
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relational database contains patient demographic data, clinical 
histories, radiologic findings, diagnoses, and examination data. This 
database is linked with digital images, a course authoring system and a 
set of tools for image display. The database contains modality- 
specific tables, diagnoses using the ACR code, and references and 
authors. A commercially available relational database, Oracle, was 
chosen, to maintain a high degree of compatibility with other 
developers and systems. For image storage and transmission, the 
ACR/NEMA 8 Digital Image and Communication Standard has been 
adopted in order to again maintain compatibility. 

In addition to the standard database, a knowledge domain or 
semantic net contains information about diseases in a structure that 
implies classification and provides relationships with other diseases, 
findings, etiologies, and subordinate entities. Figure 2 demonstrates a 
segment of such a net with each indent indicating a subordinate term. 
Anatomical Structure divides from Body Function, Descriptors, 

RADIOLOGY NET 

Anatomical Structure 

Embryonic Structure 

Abnormal embryonic structure 
Normal embryonic structure 
Mature Structure 

Abnormal Structure 

Acquired Abnormality 
abscess 
calcium deposit 
growth 

tumor 

benign tumor 

atheroma 
cyst 

malignant tumor 

Developmental Abnormality 

Normal Structure 



FIGURE 2 
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Drugs, Finding, etc. Under Anatomical Structure falls Embryonic and 
Mature Structure. Abnormal and Normal Structure divides both 
Embryonic and Mature Structure. Abnormal, under Mature, leads to 
Acquired versus Developmental, etc. This semantic net provides the 
necessary links among various terms in order to provide powerful 
searches of the data and image files. 

By having the clinical histories and radiologic findings in a 
structured database, searches are possible on any combination of these 
categories in addition to simple case selection based on diagnosis code 
such as the ACR Index. By providing standardized data and image files, 
display tools, and authoring system, applications programmers can 
build their own teaching modules while maintaining compatibility 
with other developers. Teachers will be able to build instructional 
materials without becoming programmers or requiring their 
assistance. Students can use the system in either the structured 
approach, taking a course, or the browsing mode, as described below. 

Course Authoring System 

In order to create structured learning experiences, a user- 
friendly set of tools are needed to allow the teacher to create a course. 
Such courses must provide for sophisticated branching based on 
answers provided by the student. The teacher must plan the course 
very carefully, in order to take full advantage of these systems. 
Incorporating high-resolution images into commercial authoring 
systems has turned out to be much more difficult than expected. 

Browsing Mode 

In contrast to the structured learning provided by a course 
authoring system, students often desire to simply look up a case or 
cases of a particular disease or showing certain findings. A method of 
conveniently searching the data and image files, or browsing, is 
needed to provide this flexible approach. With the semantic net and 
the relational database, such complex searches are facilitated, 
allowing the student to gain knowledge in a variety of directions the 
student may choose to explore. However, this method does not replace 
the need for structured learning that assures exposure to a certain set 
of material. 

Hardware and software considerations 
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The hardware platform chosen for this project is the MAC-II 
(or MAC llx). Peripherals have been chosen to permit migration to 
other platforms such as PS/2 and OS/2. The development system 
contains five Megabytes of RAM, a 16 Megabyte RAM-card, tape drive, 
ethernet controller, and 1200 baud modem. One 80 Megabyte disk 
stores data while a 300 Megabyte disk is used for images. Radius 
display monitors provide 1280 x 1024 pixel images with a grey 
scale depth of 8 bits. Images are stored at approximately 2000 x 
2000 x 8 bits. Most images have been obtained by digitizing xrays 
with a DuPont laser digitizer, at either 100 or 200 microns per 
pixel and 12 bits depth in grey levels. 

Macintosh Programmers Workbench "C" was chosen to create 
the display tools and linkages with the various databases and other 
modules. The Structured Query Language (SQL) interface and 
HyperCard provide the data entry screens and routines to enter cases 
into the system. 

Project Status 

The relational database has been designed, developed, 
implemented, and tested. The semantic net is still undergoing testing 
and revisions. The course authoring system is still under 
investigation, although Course of Action has been tested and partially 
implemented. The pilot study for DRILL is mammography. 
Approximately 100 cases have been entered and the images digitized. 
The use of CD-ROM is also still under investigation, with concerns for 
speed of image access causing our hesitation, as well as problems with 
standards in CD-ROM formats. 

To date one major application has been built on the DRILL 
foundation, namely a Magnetic Resonance Imaging (MRI) Simulator. 
This application not only imitates the console of a GE Signa system, 
but modifies images in real time to make them appropriate for 
various pulse sequences. This simulator is used to train technologists 
without having to use the actual scanner console, which is needed to 
scan patients 16 hours daily. 

Future Plans 

After the pilot is completed later this year, we intend to test 
the mammography module with residents and staff at the participating 
institutions. Based on success in that trial, additional modules will be 
added to expand the clinical content. The basic software will be made 
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available to interested parties for additional projects to stimulate the 
standards concept. 

Another application program planned for the DRILL foundation 
is the ICON system from Yale. 9 This expert system provides a series 
of images to assist in the differential diagnoses. Some images 
demonstrate similar findings in different diseases, while others show 
the range of findings associated with a particular disease. 

Interfaces with Picture Archive and Communications Systems 
(PACS) will be created and speech recognition hardware will be added 
to simplify the user interface, especially for the student. Porting the 
software to the IBM environment will also be considered. 
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Computer-Aided Differential Diagnosis in Radiology-Hyperray 
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INTRODUCTION 

Computer-aided instruction (CAI) has been attempted since 
the 1960's, first at the public school level, and later at the 
University level [1]. Interest in application of CAI techniques to 
medicine is growing [2-9], with much of the effort directed toward 
nursing education [10-12]. New technology in microcomputer 
software and hardware has enhanced the ability of medical 
educators to develop computer-assisted educational programs [13]. 
The number of simulation programs has also increased [14]. 
Visually oriented computers such as the Macintosh offer a new 
approach to medical CAI, in that diagrams, radiographs, scans, and 
other medical images can be utilized either directly on the 
computer screen, or in conjunction with videodisk technology [15- 
19]. 

Not all efforts in medical CAI have been successful. In one 
study, investigators found that significant improvement in quality of 
care occurred with the use of the algorithms, yet improvements 
tended to disappear quickly when physicians stopped using the 
algorithms, suggesting that the improvements were due to use of 
the algorithm and not due to an educational effect. However, 
learning may not occur if the system provides answers without also 
providing explanations. CAI programs for medical education using 
new technological advances are just beginning to be implemented. 
In particular, medical CAI programs using HyperCard, a new 
software package for the Macintosh, are in an early stage of 
development. Some examples include a chest imaging program at 
Yale created by M.A Swett, a central nervous system program by 
Carl Jaffe, also at Yale, and an anatomy program by R. Chase at 
Stanford [20]. 
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CAI offers a number of advantages over traditional teaching 
approaches. Not all students begin a course of study with the same 
background, knowledge, and skills. CAI lessons can be tailored to 
each student's needs, and allows concentration of studies in areas 
where the student is weak. 

The objectives of the project described here were: 

1. To develop a teaching aid using the Macintosh 
computer to instruct residents m techniques of 
radiological differential diagnosis; 

2. To establish a general framework which will 
allow similar systems to be established for 
other medical specialities; 

3. To evaluate the effectiveness of the teaching aid 

for resident training. 



METHODOLOGY 

The HyperCard software on the Macintosh SE was used to 
establish a framework for teaching radiological differential 
diagnosis. The resulting program, denoted HyperRay, is an early 
attempt to provide radiological differential diagnosis on a 
Macintosh computer which is available to residents in the film 
reading room. HyperCard offers the advantage of setting up 
information which is easily indexed by various keys. This allows the 
student to view the information in an hierarchical structure with 
different starting points. For example, disorders may be 
categorized by body location, such as chest, or by functional system, 
such as bone. 

The information is organized into units denoted cards. 
Each card fills a computer screen. The informational content was 
determined with the help of a radiology resident. The actual 
creation of the cards in HyperCard was done by a student assistant. 
Reference sources include several radiological textbooks [21-23]. 

Figure 1 shows a sample hierarchical structure with chest as 
the major division. The program then allows the user to branch to 
any topic at the right end of the tree. Similar hierarchies exist for 
other Dody locations. An icon is provided at the top level. Clicking 
on an indicated body area will bring up the screen at the top of the 
corresponding hierarchy. Similarly, hierarchical arrangements by 
functional systems are also utilized. 
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Figure 2 shows some sample screens for the bone subsection 
created with HyperCard. The + indicates that more information is 
available on that subject. For example, card 5 is obtained by 
clicking on "Periosteal Reactions" on card 4. This arrangement 
provides rapid access to details on specific disorders. 

The HyperCard software on the Macintosh provides an 
environment which accommodates the novice computer user while 
providing sophisticated searching capabilities. An issue which is 
still unresolved is whether this approach to teaching medical 
residents will prove more effective than traditional methods. In 
order to shed light on this issue, the HyperRay program will be 
evaluated in the resident training setting. 



EVALUATION 

The HyperRay teaching aid will be evaluated according to 
the following procedure. The residents will be tested for their 
ability to read radiographs and provide a differential diagnosis 
before using the HyperCard program. They will then be trained 
using the program. After 3 months, they will be re-tested. The 
testing will be done using a self-assessment program written by 
Radiology Information Systems Consortium for IBM-PC 
compatible computers. The program consists of 165 images and 
usually two sets of five multiple choice questions for each image. 
The computer automatically scores the test. 



FUTURE INVESTIGATION 

We plan to expand and adapt HyperRay to provide 
diagnoses covering all radiological subspecialties and, later, to 
include brief notes on specific diseases. In the final state, digitized 
radiological images will be displayed by the system. 
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Figure 1: Sample Hierarchy 
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CARD 4 



Index for Bone 

o Osteopenia + 

o Osteosclerosis + 

o Periosteal Reactions + 

o Trauma and Fractures + 

o Localized Bone Lesions + 

o Arthritides+ 

o Soft Tissue Calcifications and Ossifications + 
o Skull + 
o Orbits + 

o Nasal and Paranasal Sinuses + 
o Jaws and Teeth 
o Spine and Pelvis + 

CLICK ON ANY TERM TO GO TO THAT SUBJECT 



CARD 5 



Diseases with Periosteal Reactions 

o Malignant Bone Tumors + 
o Benign Bone Tumors + 
o Infections + 
o Arthritides+ 
o Trauma + 

o Pachydermoperiostosis 

o Vascular and Lymphatic Disease 

o Bone Infarcts 

o Eosinophilic Granuloma, Histiocytosis X 

o Gaucher's and Other Reticuloses 

o Thyroid Acropachy 

o Hypertrophic Osteoarthropathy 

o Tuberous Sclerosis 

o Infantile Cortical Hyperostosis (Caffey's Disease) 
o Fluorosis 

o Gardner's Syndrome 
o Idopathic Degenerative 



Figure 2: Sample Cards from Bone Section 
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In a visually oriented field such as diagnostic radiology, 
illustrations, diagrams, and other types of artwork can 
complement verbal description and photographs. Artwork 
can be of particular value when an anatomic structure or 
feature is established indirectly. For example, when 
intraluminal contrast material is used to outline an 
alimentary tract mass, a drawing of the lesion from outside 
the lumen or in cross-section may help the reader 
understand exactly what the contrast material is outlining 
and why the diagnostic images appear as they do. Artwork 
may also simplify the description of anatomic concepts and 
add interest and aesthetic value to a presentation. 

The medical illustrator should be involved in the 
project from an early stage. The information and concepts to 
be illustrated should be explained in detail to the artist using 
sketches, specimens, anatomic models, photographs, and so 
forth. The illustrator can then present the information and 
concepts in a visual form and create the final artwork. An 
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initial sketch is made from which successive versions are 
prepared in concert with the author. When the final version 
is satisfactory, the sketch is "inked in," resulting in a finished 
piece of artwork. 

The microcomputer is a tool that can be used to 
transform the visual concept into artwork [1]. The artist, 
using the computer, can create straight lines, smooth curves, 
regular geometric shapes, and other elements of graphic 
design. Most important is the ability to create smooth, 
curved lines of irregular path and variable weight for 
executing elaborate drawings. Alterations can be made 
rapidly and with minimal effort. Although the computer 
will not transform an artistically maladroit physician into a 
professional illustrator, in the hands of a skilled medical 
artist, the microcomputer can rival traditional artists' 
techniques such as pen-and-ink and color-and-brush in 
producing high-quality finished artwork for medical 
illustration. 

This article explains the use of microcomputers in the 
production of a series of medical illustrations for a textbook 
of skeletal radiology [2]. 

MATERIALS 

The following computer systems and peripherals were 
used in various combinations: Macintosh II, Macintosh SE, 
and Macintosh Plus computers with various hard and floppy 
disk drives (Apple Computer, Cupertino, CA); LaserWriter 
NTX, LaserWriter Plus, and Imagewriter II printers (Apple 
Computer, Cupertino, CA); and DEST PC Scan Plus image 
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scanner (digitizer) (DEST Corp, Milpitas, CA). The Macintosh 
computers have a mouse which can control the movements 
of the cursor on the monitor screen with great precision. 

The software programs that were used included 
Illustrator 88 (Adobe Systems, Mountain View, CA), 
SuperPaint (Silicon Beach Software, San Diego, CA), and 
Cricket Draw (Cricket Software, Malvern, PA). The 
descriptions of methods below reflect the use of Illustrator 88 
[3-4] running on a Macintosh II with 5 Mbyte of random 
access memory. 

CREATING THE INITIAL SKETCH 

Once the graphic concept is clear and the composition of 
the artwork has been planned, a preliminary rough sketch is 
made. This sketch may be a freehand pencil drawing, a 
tracing of a radiographic image, or a previously rendered 
drawing with adaptations. A digitized version of this 
preliminary sketch is entered into the computer by way of the 
image scanner, producing a bit-mapped image. This image is 
used as a template to prepare a line drawing. The template is 
traced over with a freehand drawing tool (pen tool) 
controlled by the mouse and then discarded, leaving only the 
drawing. An automatic tracing tool is also available where 
the computer itself prepares a line drawing of the template by 
tracing the edges. The utility of this automatic tracing tool 
depends on the intricacy of the original material and the 
quality of the digitized image. 

Radiographic images may be digitized into the computer 
by use of a video camera digitizer or by scanning high- 
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contrast photographic prints; the results from either are 
often unsatisfactory. A more practical method is to trace over 
the salient features of the radiographic image by hand, using 
a photograph or the original film, and then digitizing the 
tracing with the scanner. 

It is also possible simply to draw freehand in the 
computer using the pen tool and other drawing tools without 
using a template. Such tools are actually computer 
subroutines for creating straight lines, regular and irregular 
geometric shapes, smooth curves, and other graphic 
elements. 

MODIFYING THE DRAWING 

Once the initial sketch has been compiled, it can be edited, 
modified, and embellished. A complex drawing is built by 
layering many individual graphic elements, or objects, on top 
of each other (Fig. 1). These objects are maintained by the 
computer as mathematically described objects rather than as 
bit-maps, a strategy that allows them to be modified and 
edited by changing their attributes and parameters. For 
example, a circle may be described by its radius, the location of 
its center, and the thickness of the line at its perimeter; 
straight lines have a starting point, a direction, and a length. 
A circle may be changed in size or location, stretched into an 
oval, made opaque, or shaded to appear spherical, all within 
the computer. A line may have any location, length, or 
direction; it may changed to any thickness or weight; and it 
may be solid, patterned, or dashed. Even if layered on top of 
one another, each individual element of a drawing can be 
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selected and changed without altering any other portion of 
the drawing. One can also create composite objects that can 
be edited and modified together. Objects may be translated, 
rotated, reversed left for right, inverted top for bottom, 
stretched, duplicated, enlarged or reduced, and distorted (Figs. 
2-3). For example, to create the shadow of an object, one 
might duplicate the object, give it some angular distortion, 
make it opaque, rotate it 90°, and translate it to the base of the 
object. The view of the drawing on the computer screen can 
be zoomed in or out for detail work or for examination of the 
overall drawing. 

Text can be added to the composition in a wide variety of 
typefaces and styles; the text can be edited with almost the 
same flexibility as a word processing program. Text may also 
be used as design elements and formed into circles or bound 
along curved paths. 

PRINTING THE FINAL VERSION 

Object-oriented graphics programs permit the resolution 
and quality of the final printed output to exceed that of the 
computer monitor. PostScript (Adobe Systems) is the most 
common computer language used by high-level art and 
graphics programs to describe and manage graphic elements 
through mathematical algorithms. Printers and other 
computer output devices need a PostScript raster image 
processor to translate these mathematical algorithms; an 
output device with such a processor is said to be PostScript- 
compatible. Such a device is necessary for high-quality 
output. 
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Medial Aspect 



Figure 1. Deltoid ligament of the ankle (Illustrator using 
Macintosh II). (A) Final output printed by a Postscript- 
compatible laser-xerographic printer with 12 pixels/mm 
resolution. (B) Final version as seen on the computer 
monitor. (C) The drawing on the computer monitor as it is 
being worked on. The tool box on the left has options for 
selecting objects, moving them, zooming the view, entering 
text, and so forth. Additional menus are located along the 
top. Scroll boxes on the right and at the bottom allow views 
of different portiqns of the drawing. Reprinted from Skeletal 
Radiology: The Bare Bones by F. Chew with permission of 
Aspen Publishers, Inc., © 1989. 
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Figure 2. Pelvic fractures (Illustrator 88 on the Macintosh 
II)* This series was created from a single drawing of a 
normal pelvis that was duplicated and fractured in different 
ways. Reprinted from Skeletal Radiology: The Bare Bones 
by F. Chew with permission of Aspen Publishers, Inc., © 
1989. 
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Figure 3. Ligamentous sling of the carpus (Illustrator 88 
on the Macintosh II). The inset of the dorsal aspect was 
created by reversing the basic drawing of the bones, 
layering the triquetrum on top of the pisiform, and scaling 
the image down. The line weight and the typeface of the 
inset and the main drawing are uniform. Reprinted from 
Skeletal Radiology: The Bare Bones by F. Chew with 
permission of Aspen Publishers, Inc., © 1989. 
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The final versions of all the drawings in this project 
were printed by a PostScript-compatible laser-xerographic 
printer with 300-dpi (120 dots per cm) resolution on letter- 
size (8.5 x 11 in) 80-lb white coated stock (Vintage Velvet 
paper, Seneca Papers, Rochester, NY). The artwork was 
submitted to the publisher as 5 x 7 in (12,5 x 17,5 cm) black 
and white photographic prints. A total of 60 figure parts were 
prepared for publication. 

DISCUSSION 

Several programs for creating art are available for the 
Macintosh computer. These programs can be categorized into 
paint, draw, and illustration types. A painting program 
produces a bit-map on the computer screen. The painting can 
be edited only by erasing or being painted over. Examples of 
painting programs include MacPaint II (Claris Corp., 
Mountain View, CA) and Superpaint (Silicon Beach 
Software, San Diego, CA). A drawing program produces 
lines, circles, and other geometric shapes whose attributes 
(line weight, fill patterns, etc.) can be edited subsequent to 
their creation. Examples of drawing programs include 
Cricket Draw and MacDraw II (Claris Corp., Mountain View, 
CA). An illustration program is one that has a variety of 
tools, including freehand, drawing, and often painting tools. 
Examples of illustration programs include Illustrator 88, 
FreeHand (Aldus Corp., Seattle, WA), and Canvas (Deneba 
Software, Miami, FL). As these types of programs have 
evolved, features of drawing and painting have become 
integrated. The approximate price range for software 
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programs is from $100 to $500. Computer-aided design 
programs that are intended for mechanical or architectural 
drawing, where exact dimensioning and scaling is critical, 
may lack tools for freehand drawing, and are therefore not 
particularly suitable for medical illustration. 

In selecting a computer system for medical illustration, 
the choice is restricted to the Apple Macintosh by the 
availability of software. Computing speed is important 
because of the mathematical calculations involved each time 
objects are created or modified. Although the dimensions of 
the drawing are not limited by the size of the monitor, a 
larger screen size allows more of the work to be seen while 
drawing. The program Illustrator 88 requires at least 2 Mbytes 
of random access memory to run properly; many Macintosh 
systems are supplied with only 1 Mbyte. A hard disk drive is 
essential because of the large size of the programs and the 
large size of the files they create. Mouse substitutes such as a 
drawing tablet with a pencil-like stylus (IS/ABD Input 
System, Kurta, Phoenix, AZ) may be helpful if one wishes to 
create the initial sketch by tracing over a picture. The output 
device of choice is a laser-xerographic printer that is 
Postscript-compatible. A dot-matrix impact printer has much 
lower resolution and while it can be used for rough drafts, it 
should never used to print the final artwork. Film recorders 
may accept graphic output for making 35 mm slides. 
Illustrations from the Macintosh can be output onto a photo 
imagesetter such as a Linotronics model L300 (Linotype Co., 
Hauppauge, NY); this would allow resolution as fine as 2540 
dots per inch (100 dots per mm). 
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CONCLUSION 

The vast potential of the microcomputer for creating 
high-quality art and graphics [5-6] is being realized by the 
availability of high-level illustration programs running on 
fast computer systems. Although illustrating on the 
computer is still time-consuming, it is generally more 
efficient than using traditional artists methods. The chief 
advantage of the computer is in editing, modifying, and 
changing a drawing. The computer is not a substitute for a 
clear conceptualization of the illustration and it will not 
make a muddled concept clear. It is simply another tool the 
artist can use to put visual images on paper. 
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Development of an Authoring and Tutorial Program 

Marvin E. Goldberg and Jack Pejsa 
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Although there are several authoring 
programs available, we have developed a com- 
bination authoring and tutorial program which 
allows the author to make provision for inter- 
active instruction, as well as easy links to 
audio visual aids. 

PROGRAM DEVELOPMENT: 

The program was developed utilizing Turbo 
Prolog Version 2.0. The prolog programming 
language was chosen because of its ease in 
prototyping and the availability of windowing 
programs which facilitate the generation of 
interactive menus and dialog boxes on the 
screen. Prolog has been used as a prototyping 
language for other applications. While the 
resulting program code becomes, somewhat, 
large, it is still useful, and final develop- 
ment of the program could be carried over into 
another programming language. The program has 
been developed for the MS-DOS system and has 
run successfully on the IBM PS/2 family of 
computers. It can run from floppy disk drives, 
although if placed on a hard drive, program 
execution will be faster and smoother. 

DESCRIPTION OF PROGRAM: 

The program was developed in two parts. 
The first part, "Author", is devoted to the 
authoring program and the second part, referred 
to as "Tutor", presents the material to the 
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student. In practice, the two parts would be 
located on separate disks, and only the 
tutoring portion would be presented to the 
student for his use. 

The authoring segment of the program is 
menu driven. Text material which the author 
desires to be visible to the student is 
developed in an editing screen which has 
capabilities common to most word processors. 
Use of the system is easy for anyone with 
previous experience with word processing 
programs. When completed to the author's 
satisfaction, the text material can be saved to 
disk. Similarly, questions in the form of 
multiple choice or true/false questions, can be 
generated easily by the instructor and also 
saved in their own separate files. The 
sequence of presentation of the text screens 
and question screens can be edited by the 
instructor and rearranged at will. All such 
material can be saved to disk for further use. 
At the present time, the program does not drive 
visual aid hardware, but provision is made for 
development of an interactive module for use 
with video disk player. 

The tutorial segment, which is the only 
segment visible to the student, prompts the 
student for his name and choice of program to 
view. The program presents text screens and 
questions in the order selected by the author 
of the program. For each question screen, the 
student may elect to receive immediate feedback 
as to his correctness of answering the 
question, or can elect to proceed without such 
feedback. In any event, all responses are 
recorded by the program and at the completion 
of the session, the student is informed of the 
number of correct responses. This information 
is also permanently recorded on disk by the 
program for review at a later date by the 
instructor. 
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A Personal Computer Based User Interface 

for Viewing & Manipulating Digitized Radiographic Images 

for Instructional Applications 
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Boston, MA 



Abstract 

We have designed a user interface for the display and manipu- 
lation of radiographic images that is intended for use in digitized 
teaching files and computer aided instruction (CAI) applications. By 
making the way the viewer interacts with a set of digitized images on 
the computer screen as similar as possible to the way he interacts with 
conventional films on a viewbox we have tried to create a system that 
combines the directness of the conventional teaching file with the 
power and flexibility of CAI. 

Introduction 

The conventional film-based teaching file has long been a 
mainstay of radiology education for medical students and residents. As 
a teaching tool, it has the distinct advantages of simplicity and direct- 
ness. The teaching file is a direct link between the way the student ex- 
amines and manipulates radiographic images during training and how 
he will do so during his professional career: the images appear on ra- 
diographic film and are manipulated by moving the films around on a 
viewbox. 

Recently there has been a great deal of interest in creating 
teaching files and film libraries that are linked to a computer database 
[1,2]. Under such a system the database could be searched by criteria 
such as anatomic site, features, diagnoses or imaging modality in order 
to quickly identify relevant radiographs. Of the various methods avail- 
able for linking a computerized database to a film library, the most di- 
rect way is to create a digitized image library in which images are 
stored in digital form on magnetic or optical disk where they can be 
accessed and displayed directly on the computer screen. This brings 
the potential capabilities of rapid access to large libraries of high- 
quality images without the disadvantages of the conventional teaching 
file. 

The major tradeoff of the digitized image library versus con- 
ventional film is that the radiographs are displayed on a video display 
terminal rather than in their native form, changing the nature of the 
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interaction between the viewer and the image. Manipulations that are 
routinely done with film-based radiographs, such as brightlighting 
underpenetrated areas, circling interesting features with a grease 
pencil, or, in some cases, using a magnifying lens to study fine details, 
are not possible under most digitized image display systems. 

We feel that it is desirable to make the learning experience as 
close to actual practice as is feasible. Recently we have prototyped a 
user interface for image presentation that accomplishes this on readily 
available personal computers. 

The Interface 

There are five elements of our interface which are described 
below. An authoring and presentation system incorporating these fea- 
tures, as well as other hypermedia and knowledge management capa- 
bilities, has been developed in our laboratory. This system, known as 
Explorer-2, is implemented on the Macintosh II computer [3, 4]. 

LThe "Viewbox" 

In Figure 1, two digitized mammograms have been placed next 
to one another in a single window which acts as a virtual "viewbox". 
Like a real viewbox, the images can be picked up and moved around to 
different positions. To do so, the user positions the mouse-controlled 
cursor on the image he wishes to move, depresses the mouse button, 
and drags the image to the desired position. Releasing the mouse but- 
ton drops the image into place. This makes it easy to place PA and 
lateral films side by side or to arrange a series of films chronologically. 

2. Brightness and Contrast Controls 

Two sets of controls are associated with each image; they ap- 
pear to "float" over the image and can be moved about or made 
invisible at the user's discretion.. These controls appear as the 
rectangular areas at the top and left margins of the left-most image in 
Figure 1. 

One set of controls in each image is the pair of horizontal ar- 
row sliders shown at the top of the left-hand image. These controls 
graphically show the user the amount of contrast and brightness in the 
image and allow him to adjust these parameters as needed, changing 
the global appearance of the image. This permits selective highlighting 
of specific image features. When multiple images are displayed on the 
viewbox simultaneously each has its own brightness and contrast con- 
trols, allowing the viewer to compensate for differences in film tech- 
nique. 
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Figure 1: Elements of the user interface. The digitized images of two 
mammograms are shown side by side in a single window. The images 
can be moved around, overlapped and resized with the mouse. 
Textual and graphical information can be added to the images as 
overlays. As described in the text, the two rectangular panels in the 
upper left of the leftmost image represent controls that allow the user 
to change the contrast and brightness of the image as well as to select 
various tools with which to manipulate the images. Text and graphics 
can be overlaid on the image, as shown at the right. Blocks of text and 
hypertext can be placed elsewhere within the window, or in other 
windows. A portion of a hypertext block can be seen underneath the 
leftmost image. In the example shown here, the user has positioned a 
small "magnifying lens" cursor over a portion of the image on the 
right that contains a feature of interest (calcifications), the magnifica- 
tion of which is shown in Figure 2. 
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3. The "Magnifying Lens" 

The group of symbols or "icons" along the left border of the 
image defines a set of tools that allows the user to manipulate the im- 
age in several ways. To select a tool, the user positions the cursor on 
the desired tool and clicks on it with the mouse. 

One of the most important of these tools is the magnifying lens, 
represented iconically as a miniature hand-held lens. When the magni- 
fying lens tool is selected, the on-screen cursor that is linked to move- 
ments of the mouse turns into a small picture of a magnifier (this cur- 
sor is present in the center of the left-hand image of Figure 1). The 
user positions the cursor over some point of the image and presses the 
button on the mouse. At the moment the mouse button is depressed, a 
rectangular window pops up over the area selected and displays a 
magnified view of the image. This pop-up magnifying window is 
shown in Figure 2. While the mouse button is depressed, the user can 
move the mouse in any direction and the window containing the 
magnified image will slide over the fixed unmagnified image exactly 
as if the user were holding a true magnifying lens to the image. When 
the user releases the mouse button, the magnified image instantly 
disappears. The rapidity of the appearance and disappearance of the 
magnifying window allows the user to examine multiple regions at 
high magnification without losing his orientation and without 
performing a feat that would be "unnatural" with real film, as for 
example, making the entire image larger. 

It is important to note that the magnified image produced by 
the magnifying lens tool represents true, full-resolution image data 
rather than false magnification through interpolation or replication of 
pixel values. The data produced by commercial radiograph scanners 
has approximately four-fold greater resolution than can be displayed 
on the screen of a personal computer. The magnifying lens gives the 
user access to that high resolution data in a natural way. 

The magnifying lens tool's magnification can be adjusted by 
the user by issuing various keyboard commands, allowing the user to 
scan through the image at low magnification and zoom in to high 
magnification to see very small details. 

4. The "Bright Light" 

Depressing a designated key on the keyboard puts the magnify- 
ing lens into "bright light" mode, a setting that increases the local con- 
trast and brightness of the image shown within the lens. This approxi- 
mates the effect of holding an overpenetrated section of the film up to 
a bright light in order to provide a virtual bright light facility without 
changing the overall brightness and contrast of the entire image. Like 
the magnifying lens itself, this technique allows the user to scrutinize 
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The differential for well- circumscribed masses of 
the breast includes such benign lesions as fibroadenoma 
and tubular adenoma. Irregular borders, or retraction 
of the shin and/or parenchyma, are more associated 
with malignancy. More^d^ 
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Figure 2: In this figure, the user has pressed the mouse button while 
the "magnifying lens" cursor was on the region shown in Figure 1. A 
rectangle has popped up showing greater detail in the region the user 
selected. As the mouse is moved around, this rectangle slides over the 
image, magnifying the portion beneath it. The user can change the 
degree of magnification, so that even greater detail can be examined. 
By depressing a designated key, the user can also locally increase the 
brightness within the area of magnification, achieving a virtual 
"bright light" facility. 
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an area of interest without changing the surrounding landmarks. The 
bright light mode can be applied while the magnifying lens is 
displaying the image at increased magnification. Alternately, the 
magnifying lens can be set to "IX" (no magnification) so that the 
bright light facility can be used without adding magnification. 

5. The "Grease Pencil" 

Other icons represent a group of graphical tools that substitute 
for the grease pencil. Using these tools, the user can "mark up" the im- 
age in any arbitrary screen color, erase portions of the markings, and 
save and discard the set of markings to disk at his discretion. The 
markings are treated as a graphical overlay distinct from the actual 
image information, making it possible for different users to make 
different markings on the same image. 

Among the grease pencil tools is one that allows free hand 
markings to be made as well as an eraser for making free hand revi- 
sions. In addition to these, there are tools that allow users to draw cir- 
cles, ovals, rectangles, lines and arrows, as well as a tool to insert 
textual information such as captions and comments. 

Conclusions 

We have designed a user interface for the display and manipu- 
lation of digitized radiographic images that overcomes the problem of 
the unnaturalness of the computer-displayed image. By mimicking the 
way that people interact with film, the interface combines the power of 
the computer with the flexibility and naturalness of film reading. 

Our interface displays digitized images that look and behave 
like film. The images can be moved about and overlapped on a virtual 
viewbox, marked on with a grease pencil and other graphic tools, se- 
lectively enlarged with a virtual magnifying lens to give the user ac- 
cess to the full resolution of the original radiograph without losing his 
global orientation, and selectively brightlight areas of overpenetration. 
In addition, contrast and brightness controls allow the user to alter the 
image globally in order to highlight specific image features. It does 
this on a readily available personal computer running in a software en- 
vironment specifically designed to produce computer-aided tutorials. 
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A Radiology Hypertext System for Resident Education 
& Clinical Decision -Making 

Charles E. Kahn, Jr. 

Department of Radiology, University of Chicago, Chicago, IL 



Hypertext is a means of organizing information which allows one 
to explore a topic by utilizing the connections among various related 
pieces of information* "Fact/File" is a developmental system that 
provides hypertext access to a wide variety of information of interest to 
radiologists and radiology residents. This system has been 
implemented as a module of the MARS radiology information system at 
the University of Chicago Medical Center. 

The system is organized as a collection of "frames/' Each frame 
consists of one computer screen of textual information about a specific 
topic, such as a disease entity or a radiological differential diagnosis. 
Each frame is indexed alphabetically by keywords in its title. To view 
a particular frame, the user enters a word or part of a word from the 
frame's title. By entering only one letter, the user can view all frames 
whose titles have words beginning with that letter. 

The frames also can be accessed through a hierarchically arranged 
topical index. In the first level of the index are the organ systems, such 
as "Gastrointestinal (GI) system." If the user selects this topic, a sub- 
index appears; this menu contains choices such as esophagus, stomach, 
liver and pancreas. If the user selects "stomach" from this list, the user 
can choose from categories such as congenital disorders, neoplastic 
disease, and inflammatory disease. 

One can create 'links" between frames which allow a user to move 
easily among related frames. For example, while exploring the 
differential diagnosis of "thickened gastric folds," the user can move 
immediately to the discussion of Menetrier disease without returning to 
the index. The user can then return to the previous frame (the default), 
pursue another sub-topic, or exit to the index. 
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The system presently contains about 800 frames which discuss 
radiological gamuts as well as relevant anatomy, physiology and 
pathology. The data are very compact, and require less than 250 
kilobytes of storage. The programs are written in the ANSI-standard 
MUMPS programming language. Radiology residents and attendings 
can use the system at all times from terminals located throughout the 
radiology department. Information is retrieved and displayed 
instantaneously. 

The Fact/File system is a simple, MUMPS-based hypertext system 
that can be integrated into a clinical radiology information system. 
Because physicians can explore the system without leaving their work 
area, the system is useful as a desktop reference and an educational 
tool, particularly for radiology residents. 



MAMMO/ICON: Voice Activated Intelligent Radiologic 
Image Display 

Paul R. Fisher, Pradeep G. Mutalik, Ian Maddison, et al 
Department of Diagnostic Radiology, Yale University School of Medicine, 
New Haven, CT 



ABSTRACT 

An integrated system for computer-aided diagnosis and reporting of 
mammographic examinations has been developed, utilizing both an image 
database and a rule-based medical advice generator* MAMMO/ICON is de- 
signed to minimize the radiologist's time and effort, by automatically delivering 
context sensitive advice as a natural byproduct of the report dictation process. 

To use MAMMO/ICON, case findings and preliminary diagnostic 
conclusions are dictated into a voice recognition report transcription system. 
This generates a standard radiology report that can be printed right away. At 
the same time, clinical and radiologic findings are passed to MAMMO/ICON 
for analysis. 

MAMMO/ICON works in the background to produce two fundamen- 
tally different forms of advice. Written text includes evidence in favor of, or 
against the radiologist's hypothesis or other competing diagnoses, in computer- 
generated prose, immediately after the case has been dictated. The output 
format depends directly on the degree of agreement between MAMMO/ICON^ 
analysis and the radiologist's conclusion: significant differences are highlighted 
with supporting evidence, while agreements are tersely noted. Selected images 
are displayed using a modified form of IMAGE/ICON, which displays known 
examples of each diagnosis being considered, as well as a dynamic atlas of 
radiologic features, organized along one dimensional maps called axes. 

We believe that successful computer-based advice systems in radiology 
will deliver relevant information at the time of reporting, when it is most useful. 
Mammography is a growing portion of our radiologic workload, and involves 
subtle feature detection and expertise in interpretation. MAMMO/ICON 
represents one approach to develop tools that facilitate proper film interpreta- 
tion in this challenging field. 

INTRODUCTION 

As the modern practice of Radiology grows more complex, clinical radiolo- 
gists find they are overwhelmed by the amount of information available in, for 
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example, journals and new texts. The addition of new imaging modalities has only 
added to the burden of the radiologist who is trying to keep abreast of the relevant 
literature. This has fostered the development of sub-specialities within radiology, but 
even sub-specialists find daunting the effort needed to practice state of the art 
radiology. 

The demand for mammographic services has exploded over the past ten 
years, as clinicians become aware of the utility of early, pre-clinical detection of 
breast cancer, and the excellent benefit/risk ratio achieved by modern film-screen and 
xeromammography technique(l). However, only a small fraction of the women who 
fit the American Cancer Society guidelines for regular screening are currently being 
screened; as that portion grows, mammography will assume a larger portion of the 
radiology work schedule. In addition to stand-alone breast imaging centers and 
mobile vans, group practices and hospitals are expanding their mammographic 
services, and thus more radiologists will be required to interpret mammograms. 
Mammographic expertise will thus be in demand, especially since errors in mam- 
mographic reading are particularly costly- whether they be errors of over-sensitivity 
(and thus too many unnecessary biopsies) or over-specificity ( and missing some 
cancers within their early, potentially curable phase). This demographic shift is 
reflected in the recent addition of a mammography section in the American Board of 
Radiology oral examination. 

Mammographic interpretation involves subtle feature extraction, complex 
characterization and localization of abnormal features, and coherent decision-making 
based on those features, and thus expertise is typically attained only after considerable 
clinical experience. Recent developments in artificial intelligence techniques offer the 
potential to design tools to assist clinical decision-making, and thus permit a real- 
tively inexperienced radiologist to perform radiologic interpretation at a consistently 
high level of competence(2). ICON/LPD is such a system designed for the diagnosis 
of chest film abnormalities in patients with lymphoproliferative disease(3). This paper 
will describe MAMMO/ICON, an expert system based our our experience with 
ICON/LPD, designed to assist in the interpretation of mammographic images. 

One key stumbling block to the development and use of medical expert 
systems has been that they typically make the physician spend precious time and 
effort to utilize them; in general, it takes several minutes to type in a description of a 
case, and often the system asks for information in an unfamiliar form, or requires a 
search through the patient's medical records. Physicians in general do not have the 
luxury of leisurely pondering their routine cases, and thus may be willing to tolerate a 
low (albeit non-zero) level of uncertainty in individual cases, in order to keep up with 
a demanding caseload. For example, a radiologist may not recall the specific chest 
film findings seen with paragoniasis, but when confronted with such a clinical 
question, will often not feel the need to consult reference material if the chest film 
appears normal, or is consistent with his or her general understanding of parasitic 
disease in the chest. We propose that expert systems such as the ICON series will be 
accepted into the day to day clinical practice of radiology only when they require 
essentially no extra time and effort by the radiologist, and that they offer their 
information as the case is being reviewed and dictated by the radiologist. 
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Recently, a number of commercially available voice recognition systems 
have been placed on the market, some designed specifically for medical transcription. 
These systems are trained by the physician to recognize his or her own voice, from a 
vocabulary of medical terms. Some systems allow word for word transcription, and 
some also employ "trigger phrases", which are predetermined abbreviations for blocks 
of text. For example, the phrase "Normal chest" might be translated into "The heart, 
mediastinum, lungs and bony thorax are normal." MAMMO/ICON uses such trigger 
phrases to smoothly interface with the radiologist; as a case is dictated into the 
transcription system, the trigger phrases are sent to the expert system for anaylsis. Its 
conclusions are displayed, available at a glance if the radiologist so chooses, and 
requires no additional effort or time from the busy clinician. For routine cases, the 
expert system may be ignored, and since it resides in a separate computer, it does not 
affect the performance of the medical transcription system. In difficult cases, the 
radiologist may directly query the expert system, such as to display known examples 
of a given diagnosis, or to see the differential diagnosis of a particular radiologic 
finding. 



SYSTEM ARCHITECTURE 

MAMMO/ICON contains three program modules, which are linked to each other. 
The voice recognition module is a commercial medical transcription system (Voice- 
Rad, Kurzweil Applied Intelligence, Waltham, MA), and which runs on a 386-based 
microcomputer. The other two modules run concurrently on a second microcomputer 
(Macintosh II; Apple Computer, Cupertino, CA). The two microcomputers are linked 
by a serial port cable. The system requires 4 megabytes of memory, and is configured 
to use two high resolution 19-inch color monitors (1024 X 768 X 8 bits of display for 
each monitor.) The images were digitized by means of a Konika film digitizer, and by 
a 24-bit color frame grabber board (RasterOps). Images and the knowledge base are 
stored on a magnetic hard disk, and on a WORM drive (Write Once, Read Many 
optical drive). 

The Image display module is written in the C language (MPW C; Apple 
Computer), and the Critique module was written in Lisp (Allegro Common Lisp, 
Coral Software, Cambridge, MA) The software interface for connection of the voice 
transcription system and the expert system microcomputers includes a commercial 
utility (PC Anywhere). 



SAMPLE OUTPUT 

MAMMO/ICON was designed based on our experience with IMAGE/ICON, 
which has been described previously(3,4). Many of the system features are derived 
directly from IMAGE/ICON; this paper will focus primarily on those features unique 
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to MAMMO/ICON. 

The rules-based expert system in MAMMO/ICON first analyzes the demo- 
graphic data of the case, and determines if that patient is in a high, intermediate, or 
low risk category for developing breast cancer. For example, advanced age is associ- 
ated with breast cancer development, so an elderly woman is, a priori, at an increased 
risk, whereas women under 30 rarely develop breast cancer (unless there is a strong 
family history). One the risk category is determined, the radiographic features are 
analyzed, and the diagnostic hypothesis offered by the clinician is compared to the 
expert system output As described previously, MAMMO/ICON assembles fragments 
of prose (from the set of rules whose predicates were satisfied by the facts of the 
present case) into coherent sentences, paragraphs, and themes. This assembly is 
performed by PROSENET (5), a set of utilities designed for text generation, based on 
augmented transition networks. MAMMO/ICON also selects a set of images from a 
large image database, which are related in some critical way with the case at hand, 
and displayed in several different modes. The most powerful mode is AXIS Display, 
in which images are aligned in rows, which have selected and ordered the images 
according to a set of well-defined criteria, such as the differential diagnosis of a given 
case, ordered from malignant to benign. We currently have 5 display axes, including 
the HYPOTHESIS axis (which illustrates proven cases of the clinician's hypothesis 
which are similar to the present case), SPECTRUM axis (which illustrates the entire 
range of appearances of the hypothesized diagnosis), BENIGN/MALIGNANT axis, 
(which order diagnoses based on their malignant potential), and SIZE/EXT. axis, 
(which orders lesion by their size), as well as the DIFFERENTIAL DIAGNOSIS axis. 
The radiologist can easily change the hypothesis on the case, and generate a new set 
of reference images. 

For example, an elderly patient obtains a mammogram, which demonstrates 
a dense stellate mass, with associated nipple retraction, and adjacent skin thickening. 
The radiologist could dictate this case using trigger phrases, which would generate a 
standard mammographic report. At the same time, MAMMO/ICON would examine 
the trigger phrase content, and display a critique of the case, based on its set of rules 
in this domain, as illustrated in figure one. The text of the critique reads: 

I 1 

. While Mammo-icon will attempt to illustrate teaching points relevant to the case 
I you have entered, it's output must be viewed only as a general guide; and good clinical 
| judgement is, as always, the key to proper radiologic diagnosis. 

| Because this patient has significant risk factors for breast carcinoma, this critique is 
I being modified to reflect her increased risk. Any woman with a positive personal 
| history of breast carcinoma; a close relative with breast ca, or any elderly woman should 
| be considered to be at increased risk. 

| You have suggested carcinoma as your primary diagnosis in the given case. 
I Mammo-icon has located relevant facts on the following diagnoses (based on your 
! patient's clinical and radiographic features): carcinoma and chf. According toMammo- 
I icon*s analysis; the diagnosis of carcinoma is strongly supported by the following 
| factors. 

I ^. I 
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First, a dense mass with un sharp or speculated borders is always suspicious for a 
malignancy, and since your patient is in a high risk group, you must assume it is 

Second, your patient demonstrates skin thickening associated with a mass. Malig- 
nant tumors can often do so by causing lymphatic obstruction. 



nam tumors can often do so by causing lymphatic obstruction. 

Third, nipple retraction, as in this case, may frequently be caused by a malignant 
process. 



I 

After a general introductory paragraph, the critique assesses the patient's risk factors, 
and assigns her to a risk category, in this case high risk. The next paragraph summa- 
rizes the radiologist's hypothesized diagnosis, and whether MAMMO/ICON agrees 
with the hypothesis. The critique then explicitly recaps the reasoning behind its 
analysis, by reporting those rules that influenced the current case analysis. 



Bnwi tnia psbwft tw ^writaant Hi Um tar li*^ 



ftfamnoHnon hH located rdovnt te±a an fnt ftitartrjg d 




Figure 1. Text output of MAMMO/ICON 

Images are then selected to illustrate the system analysis, as demonstrated in figure 2. 
Note that a classic image of a stellate mass with associated skin thickening is dis- 
played as a full-size image, since it was the most similar to the described case, and 
each of the five axes has many images to display. On the other hand, if a low density 
mass with a clear capsule is dictated, the axis display, as shown in figure 3, would be 
much different; only 2 images appear in the DIFFERENTIAL DIAGNOSIS axis, 
because the only reasonable diagnostic possibilities are lipomas and oil cysts. 
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Figure 2. Image output of case #1 




Figure 3. Image output of case #2 
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CONCLUSION 

The MAMMO/ICON system demonstrates that expert systems for assisting 
in medical decision-making may be designed to fit unobtrusively in the workday 
patterns of clinicians, so that with a minimum of effort and time they may render 
assistance, even as the case is being dictated. As we have gained experience with 
expert system development, it became clear that the field of Medical Artificial 
Intelligence has at least two hurdles to clear to gain full acceptance for our projects. 
First of all, these systems must be medically accurate and reliable. No less impor- 
tantly, the interface must be as automatic and clear as possible, or the computer 
terminal will sit unused in a corner of a room. Our guiding metaphor was that of a 
friendly colleague, blessed with perfect memory, who sits by our side as we dictate, 
and to whom we can turn, in moments of uncertainty for a quick consultation. 
MAMMO/ICON demonstrates that this is technically feasible at present, and we 
predict that this approach can only become more effective as new, improved voice 
recognition systems become available. 
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Abstract 

This paper describes the development of TNM v2.0, a HyperCard stack 
that is used to track the staging of lung cancer patients at the Medical 
Center Hospital of Vermont. In its first iteration the program consisted 
of a single HyperCard card that contained five fields and a number of 
buttons. Functions concerned with maintaining the stack and with 
producing reports were performed manually prior to the development of 
TNM v2.0. With the development of the second generation of this stack 
these functions have been automated and delegated to buttons. The 
user interface was "tuned'* to allow a graphic-user interface (GUI) that 
is more "Mac" and HyperCard-like to make use of the program easier 
for those already familiar with the Macintosh GUI. Once a basic 
Macintosh HyperCard version of the program was developed, the GUI 
was improved to provide the user with feedback in the various steps of 
the Lung Cancer Staging work-up. When the user is performing a 
TNM analysis context-sensitive help regarding the data that make up a 
particular T, N or M class is available. Diagrams demonstrating the 
sites of the various lymph nodes on CT are available to assist with the 
correct nodal classification. A final portion of this program, which 
assists in the day-to-day work-up of a lung cancer staging, involves the 
generation of reports. Prior to this version of the program, reports 
were generated by simply printing out each patient card relevant to that 
week's lung tumor committee meeting. As work on this HyperCard 
Lung Cancer Staging database system progresses, more of the 
Macintosh GUI will be utilized to assist the radiologists, medical 
oncologists, radiation oncologists, pulmonologists and radiology 
residents in their day-to-day work with staging and follow-up on these 
patients. The final stage of the project will involve networking all of the 
clinicians, pathologists and radiologists who participate in the 
multidisciplinary staging. 
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Background 

The workup of lung cancer patients in the Radiology Department at the 
Medical Center Hospital of Vermont is tracked in a HyperCard database 
that we refer to as TNM version x.x.x. The name comes from the first 
letters of the words Tumor, Nodes, Metastasis and the term is used 
throughout the field of medicine to refer to the staging of a cancer and 
the version x.x.x refers to version number of the program in use at the 
time. There are different TNM classification schemes for each of the 
different varieties of cancer. A patient is entered into the database as 
soon as one of the chest radiologists becomes aware of the patient with a 
possible lung tumor. From this point on the entire workup of the patient 
can be tracked. The TNM database program can generate reports for 
use in the radiology department, for the attendings involved in the care 
of the patient, for the tumor registry and for use in the weekly lung 
tumor conference. The program allows for data entty to be performed 
by using many of the HyperCard metaphors that the user should be 
familiar with from having used a Macintosh before. Staging of the 
patient* s tumor can be performed with assistance from the computer 
with help being available for each T, N, and M that is used to generate 
the final stage. Some form of the program has been in use for over a 
year in the Radiology Department at the Medical Center Hospital of 
Vermont and the current version has been in use since December of 
1989. The program is used to assist in coordinating the activities of the 
multidisciplinary lung cancer staging and treatment group that meets 
each Tuesday morning. 

Method 

TNM was developed using the HyperCard programming environment 
from Apple Computer, Inc. External commands and external functions 
(XCMD's and XFCN's) are used to give the program the added 
capabilities needed to generate reports and for some portions of the user 
interface, such as the displaying of dialogs. TNM uses a linked list of 
data cards to maintain the patient database. Generic navigation cards are 
used to steer the user through the database. Moving through the 
database or stack is performed by clicking on buttons and whenever 
possible standard or familiar icons were selected. 

The linked list structure was chosen because it was fairly easy to set up 
using HyperCard. The HyperTalk script that sets up the linked list is 
located at the stack level and a portion of this script is displayed in 
Figure Six. The linked list is maintained by storing the ID of the 
previous card and the next card in a hidden field called linked-list which 
is present on each card. The ID of the previous card is placed in the first 
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line and the ID of the next card is placed in the second line. This could 
easily have been done using the first and second items of the first line or 
some other variation of storing the card ID as a variable within a hidden 
field. At present the linked list is not sorted and a master list of 
members of the linked list is not maintained. If search time ever became 
too great these may represent options that could increase performance of 
the system. The last card in the patients set of data cards informs the 
user that no further data cards are available for that patient. All data for 
a patient is written onto a shared background of one of three different 
cards. The first screen seen by the user when opening the program is 
shown below in Figure Two. The user has initial control of the 
database from the panel of buttons; choices are made by "clicking on a 
button. " Clicking on a button refers to the process of placing the icon 
representing the mouse on the button and then pressing the button one 
time. Each button has a program or HyperTalk script associated with it. 
The names of the buttons are self-explanatory in most cases. AM- 
Report refers to the Tuesday morning multi-disciplinary lung tumor 
conference where members of the Medical Oncology, Pulmonary 
Medicine, Radiation Oncology, Thoracic Surgery and Radiology 
departments meet to discuss the current lung tumor cases. The Film list 
button generates the film pull list that is used by the film library staff to 
pull the films for set-up on an alternator for the Tuesday morning 
multidisciplinary lung cancer staging and treatment conference where 
members of the Medical Oncology, Pulmonary Medicine, Radiation 
Oncology, Thoracic Surgery and Radiology departments meet to discuss 
the current lung tumor cases. 
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Figure One 
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The buttons in the lower right-hand corner allow the user to get help, 
turn menus off or on, and return to the previous card. These functions 
are symbolized by standard HyperCard icons. 



JNM i»3.2.2 I 



Tumor Registry 



opflMgM 1990 J«nei H*r ni | - Ho hi n . M.D. * Ptler A, PietMeh, MO. 
ftcp.rlmenl .f Ratfiiloqu:, Medic. I Cuter H.iplt.l of VtrnDnt 
University of Yermi.l, C»llcqo *f Medicine 
flufliaBton. VT G54CM 

(~Neu; Paiient) 
( nnrl Patient) 
( RM- BtfparQ 
(~Summaiy 
{ PHD reporU ) 
(pnlienl report) 



( incompleles 

C 

c 



Dull 



Figure Two 

When the user selects the New Patient button the user is prompted to 
enter the patient's name, medical record number and date of birth. After 
these have been entered the user gets the first patient data card or screen 
which is presented below in Figure Three. This card acts as the 
summary card for the patient* s current workup. Patient identity 
information is entered by the software into each identity field present on 
each of the patient's data cards. There are three fields for entering the 
names of referring or primary doctors caring for the patient. These 
function either as pop-up menus or as text fields. This type of user 
interface for data entry was chosen because many of the patients are 
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referred from a fairly limited group of physicians. These same 
physicians are listed in the physician database that the program uses to 
generate the addresses for the physician reports. In the bottom right- 
hand corner of the card displayed in Figure Three are a number of 
buttons. Each button was selected in an effort to have the icon present 
on it represent the function which it performs. From left to right these 
buttons perform the following: printer button brings a dialog box asking 
which of three different types of report you would like to print for this 
patient; writing hand button brings up the dialog to add this patient to the 
film lists; the left and right arrows move you to the previous or next card 
containing this patient's workup; and the return button has a variable 
function depending on where in the data entry process you select it. On 
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Figure Four 



the patient data entry cards it returns you to the start-up card; from the 
start-up, TNM assist or Film List cards it will return you to the previous 
card. The buttons in the Work-up tracking of section of this card bring 
up a series of dialogs that answer questions relevant to the workup. 
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These range from asking the date a study was performed to whether or 
not useful data was obtained from the study. Figures Four and Five 
show the two workup cards that make up a patient database. Navigation 
buttons are shared between these two cards and they perform the same 
function. The TNM button brings up the card shown in Figure Six. 
This card is used to assist with completing the TNM and staging of the 
patient's lung tumor. The user clicks on the proper T t N, or M button 
and the computer places that value in the T, N, or M field on the bottom 
portion of the card. The button Stage by Computer runs a computer 
staging algorithm when it is clicked. The user is then presented with a 
dialog asking whether or not the staging information generated by the 
computer should be placed in the doctor staging field. The data in the T, 
N, M, and the doctor staging field is then transferred to the blanks on 
the patients clinical data form for cTNM, if TNM was selected from the 
clinical card, or in sTNM and pTNM, if TNM was selected from the 
surgical/pathology card, along with the relevant staging information. 
The left, rightaxid return arrows function as described above. Each of 
the choices possible for T, N> or M has a question mark button located 
next to it on the screen. This question mark button is used to allow the 
user to view the current definition of the T, N, or M at the time that the 
program was put together. Revisions in these definitions would require 
revising the help and most likely the staging that results from selecting a 
particular combination of T, N, or M. Near the mid-portion of the card 
is a section labeled Zones. This is present to assist the user in 
remembering to enter the distribution of lymph nodes involved in the 
tumor. User entries from the Zones button are placed at the end of the 
line of text in the N field on the TNM card. Data present in these fields 
is transferred to the relevant card along with the rest of the staging data 
when the user presses the Return button located in the lower right-hand 
corner of the screen. TNM stores all of the data regarding a patient* s 
workup on three cards. Each card is copied from a master set of cards 
having a different background. After being copied the ID number of the 
card is then placed in the linked-list hidden field present on each of the 
three cards, as described above. Only the information that is specific to 
each patient is different stored in the foreground. Scripts that 
an be called from one or more places in the program are all located at the 
stack level. This was done in an effort to conserve code space. The 
final portion of the program to be described here is the Film-list. A 
sample film list with the patient information blanked out is presented in 
Figure Seven. The film list is generated so that the film library staff 
knows which films need to pulled from the shelves and placed on the 
alternator for review at the Tuesday morning lung tumor conference. At 
the same time the film list also maintains an abbreviated record of the 
workup of the patient. This is useful to the film librarians because it 
provides a fairly accurate portrayal of the studies that the patient is 
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undergoing in case the film is not located where the Hospital's 
Radiology Film Library Database says it should be located. Data in each 
patient's section of the film list also provides a brief summary of the 
current status of the patients workup as well. The printout produced by 
the film list printout handler or procedure also serves as a useful tool for 
collecting additional data for entry into the database as a result of the 
conference. 
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Figure Seven 



The film pull-list is also used by the radiologist running the Tuesday 
morning multidisciplinary lung cancer staging and treatment group as a 
means of having the patient data handy in a useful format. Complete 
data on the patient would be available either from the paper record or 
from the database. 

Summary 

TNM is a HyperCard database written to track the staging workup of 
patients with lung tumors. The program assists the user in data entry by 
reducing the amount of typing necessary to enter data and by using 
HyperCard metaphors to ease the data entry process. A multitude of 
enhancements to the current version of the program are planned and 
these include: production of summary reports, downloading of 
pathology and other reports from the Hospital* s main frame, tutorials on 
staging and working up a lung tumor, an optical database to assist in 
staging, networking of the members of the multidisciplinary staging and 
treatment group and further enhancements to the report generating 
XCMD's and XFCN's currently being used. The program will be made 
available from the Chest Radiology Division where requests for the 
software should be directed. 
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PULSE-ECHO ultrasonics has become, in recent years, an important and widely 
accepted method for non-invasive imaging of the human body, especially for 
fetal, cardiac, and abdominal applications, and appears to offer great potential for 
further developments in diagnostic medicine. Results in varied clinical settings 
have shown medical ultrasound to be an accurate, versatile, and inexpensive 
technique yielding cross-sectional tomographic images with negligible risk or 
discomfort and with reasonable resolution. 

The present success of ultrasound has been realized through the utilization of 
only a small portion of the information actually available in an echo waveform. 
All ultrasound systems now in routine clinical service provide data on only two 
tissue properties: back-scattered echo strength and the average attenuation in the 
medium, neither of which is separated experimentally and neither of which is 
measured quantitatively. That is, present systems are based on envelope 
detection methods and therefore are capable of displaying only echo-amplitude 
(i.e. intensity) information. Thus, a conventional A-scan displays the intensity 
of the echo waveform as a function of acoustic travel time (i.e. depth) and a 
conventional B-scan produces an ultrasonic mapping in which the spot intensity 
of the display is related to the echo intensity. 

Phase information (such as the sign or polarity of the echo waveform) is recorded 
by the transducer, which is a pressure sensitive device, but is not utilized in 
present display or measurement schemes. Other parameters such as the spectral 
characteristics of the echo waveform are also easily obtained but are not utilized 
in present ultrasound systems. 

Because the interaction between sound and even simple tissue structures is an 
exceedingly complex process, a vast amount of additional information, besides 
the echo intensity, is potentially available from the ultrasonic waveform. In 
principle, in- vivo techniques could be devised which would extract and separate 
the medically significant features of the ultrasound interactions with tissue and 
would display ultrasonic tissue signatures appropriate for a differential diagnosis. 
The development of such quantitative techniques for the measurement of 
ultrasonic tissue parameters and/or the display of ultrasonic tissue signatures is 
now receiving considerable attention from the research community (1-4). 
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Of course, the mere realization that not all measured data are used for image 
construction does not, in itself, guarantee that the discarded data contain new or 
non-redundant information. Occasionally, there is also an understandable 
confusion between the achievement of a "better 1 (e.g. higher resolution, more 
displayed dynamic range, etc) image, and the development of techniques that 
generate perhaps even rather poor images with an intrinsically different (tissue) 
information content. This last point can be illustrated without attempting to 
accurately define the concept of "information" displayed: there is an intuitive 
appreciation of the differences between a high and low resolution ultrasound 
reflectiveity map of tissue (as crudely displayed in a conventional B-scan) on the 
one hand, and a reflectively map and an attenuation image, on the other. The 
former two attempt to portray the same intrinsic tissue information, but to 
different degrees of fidelity. The latter two differ markedly in their "information 
content" quite irrespective of the relative resolution, or fidelity, to which they 
may individually be obtained. 

Projections of the ultimate utility of future information extraction techniques 
have ranged from the ludicrously optimistic to the crushingly pessimistic. It is 
our contention that a realistic assessment may be made by looking to the 
physics of the ultrasound/tissue interaction. In this manner, a better appreciation 
of the range of information amenable to probing may be obtained. This is a vast 
field for investigation, and we shall heavily circumscribe our treatment What 
cannot be answered here is the medical utility of the extracted information: this 
is a problem that only protracted clinical trials can resolve. 

Ultrasound information Extraction 

The basic medical ultrasound imaging "experiment" may be essentially described 
as follows: an input ultrasound wave is generated, which is allowed to penetrate 
into tissue, where it interacts. Waves generated by this interaction, including the 
modified, emergent incident wave are detected outside the tissue. The aim is to 
make some statement about the tissue from measurements on (a subset of) the 
emergent waves ("OUT") and a knowledge of the incident wave ("IN"). Tissue is 
thus an "OUT.IN" relationships. This description is virtually identical to that 
used to define scattering experiments and it is immediately clear that scattering 
theory will play a fundamental role in our analysis. 

Given the fact that it is a medical ultrasound investigation, some details can be 
added to the above description. The input field is likely to be a longitudinal 
pressure wave, with a frequency in the 1-10 MHz range; for most applications, a 
pulsed field will be employed— broad band, but with a central frequency well 
within the indicated range. In principle, there is no difficulty in assuming the 
incident wave to be accurately known. There can be less certainty about 
specifying the (acoustic) nature of tissue. It is clearly an inhomogeneous 
medium, varying in both space and time, the latter often manifesting itself as an 
approximately cyclic variation of properties. Tissue is very richly structured, 
with spatial correlation lengths spanning a wide spectrum, well beyond the 
wavelength scales of the so-called "diagnostic" frequency range. Ultrasound 
attenuates fairly rapidly within most soft tissues at a rate that is definitely 
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frequency-dependent Since there is some experimental evidence to support the 
idea that shear waves do not propagate over significant distances within soft 
tissues, it is probably reasonable to describe the wave as a propagating pressure 
wave. Experimental techniques are not, in general, sufficiently sophisticated to 
unambiguously detect non-linear effects within tissues at the moderate power 
levels and frequencies employed for imaging, and there is thus no compelling 
reason, at present, to resort to the added complexity of non-linear theories. Soft 
tissues certainly scatter ultrasound waves and it is known that scatter-generating 
structures consist of fluctuations in at least the density, compressibility and 
ultrasound absorption parameters. The relative strength of scattering from these 
sources, as well as the overall scattering strength, are not known, but it is 
generally assumed (if only for computational convenience) that the scattering is, 
in some sense, weak. Soft tissues exhibit a complicated composition and a 
complicated thermodynamics, and, in some cases at least, are clearly not 
isotropic media: the significance of these factors has not been fully assessed in 
most applications and we disregard them, for simplicity in the following. Many 
of the acoustical properties of soft tissue change with disease, and this ultimate 
diagnostic potential of ultrasound provides one of the most exciting areas for 
investigation. 

It seems clear that there are basically three classes of information that can be 
extracted about soft tissues via ultrasound probing: some statement can be made 
about tissue acoustical properties (such as attenuation, impedance, etc.), or about 
tissue structure (i.e. the spatial distribution of th e acoustic parameters), or about 
tissue dynamics (i.e. the temporal distribution of the acoustic parameters). In 
practice, methods for achieving that end may be conveniently classified into three 
types of technique: non-quantitative imaging, tissue characterization, and 
quantitative imaging. We now discuss each of these techniques with particular 
emphasis on quantitative imaging. 

Non-Quantiativs imaging 

Conventional pulse-echo imaging maps the location, but not the type, of the 
scattering (reflecting) structures. In all commercial ultrasonic imaging systems, 
the backscattered echo sequences are envelope-detected, with amplitudes (which 
can range well over 100 dB) compressed into the dynamic range of the human 
eye and conventional display devices. Image frame rates with real-time systems 
are sufficiently high to allow spatial movements of structures to be studied, 
either qualitatively, or more quantitatively by using special (in fact, simpler) 
techniques such as M-mode recording. The typical pulse-echo image is an 
intensity map, devoid of phase or spectral information, with brightness related 
only very crudely to scatter strength. Attempts have been made at artifact 
reduction, gray-scale reallocation, resolution enhancement and utilization of 
multiple images in order to clarify information latent in the image. Most of 
these attempts are foredoomed to failure, since they neglect to consider that 
tissue information displayed in a pulse-echo image is corrupted by scanner design 
and signal processing options, as well as by machine settings. Many non- 
conventional pulse-echo imaging techniques also fall into this category by virtue 
of the fact that they produce a non-quantitative image, although sometimes based 
on an information content other than the echo intensity. Such qualitative 
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imaging schemes include (1) various methods to incorporate spectral information 
into a B-mode format, such as spectra-color ultrasonography and the so-called 
dual-frequency scanning, (2) methods to incorporate phase information into a B- 
mode format; and (3) various novel techniques for interrogating tissue and 
receiving the echo waveform. 

All of the methods just mentioned, regardless of information content or display 
technique, produce qualitative images, i.e. images in which no single parameter 
or combination of parameters is recorded or measured in a quantitative and/or 
absolute fashion. 

Although conventional (qualitative) pulse-echo imaging has become an 
important imaging modality for diagnostic medicine, it is clear that further 
advances will largely depend upon the quantitative utilization and display of the 
ultrasound information. 

Tissue Characterisation 

Ultrasonic tissue characterization includes all techniques and methods which seek 
to extract and separate from an interrogating ultrasonic wave the medically 
significant features of the ultrasound interactions and to display ultrasonic tissue 
signatures appropriate for a differential diagnosis or some other medical purpose. 
To be useful or significant, such signatures or measures must be related to tissue 
pathology or physiology. Tissue characterization is broad in scope and, in certain 
cases, can include aspects of both qualitative imaging and quantitative imaging. 
At its simplest, tissue characterization involves merely a re-display or 
recombination of pulse-echo image information in a way that is more pathology 
specific. Such simple approaches are likely to be of limited value, since they 
also tend to be machine (and operator) specific. In its most useful form, tissue 
characterization extracts a quantitative and intrinsic measure describing the spatial 
and/or temporal variations of any mixture (known or unknown) of tissue 
properties. The interactiveness between, but distinct separateness of, quantitative 
imaging and tissue characterization should be evident. Thus, observations in 
tissue characterization may lead to new imaging schemes which, in turn, may 
lead to additional tissue characterization. In a sense, quantitative imaging may 
well be the precursor to truly successful tissue characterization. 

It is convenient to separate tissue characterization into the three general 
approaches which can be taken: parameter estimation, structure characterization, 
and dynamic characterization. Parameter estimation seeks to measure (or 
estimate) the value of a particular acoustical property, such as impedance, 
attenuation, or velocity. The estimate, of necessity, provides an averaged or 
mean value of the parameter over some region, which can vary in size from the 
dimensions of the interrogating pulse to a complete A-line. In many cases, the 
parameter is assessed with respect to one or more variables (for example, 
attenuation can be measured as a function of frequency). 

Structure characterization seeks to extract some index or signature which 
characterizes the spatial arrangement of small-scale scattering structures, or their 
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strengths. Clearly, such signatures must correlate with tissue type and/or 
pathology. In practice, structure characterization often degenerates into a search 
for structure in the echo waveform rather than structure in the tissue, making the 
relationship to tissue structure much more nebulous. 

Finally, dynamic characterization seeks to characterize the time-dependent 
behaviour of tissue. Ultrasound methods are particularly suited to this type of 
approach being capable of monitoring tissue responses and motions on a rapid 
time scale. 

Quantitative Imaging 

We use the term quantitative imaging to refer to those techniques which map the 
distribution and strength of a single ultrasound/tissue interaction parameter (or, 
at least, a well defined, invariant combination of such parameters), in a truly 
quantitative way. These methods are largely experimental at present, especially 
those relying on scattered signals for input. 

Conventional pulse-echo B-mode images, despite their medical usefulness, are 
not quantitative images for two reasons. First, their image densities depend rather 
strongly on system settings and the choice of transducer. Second, even if such 
system artifacts could be allowed for, the resultant image would essentially be a 
reflectively map of tissue. Unfortunately, reflectively is a complicated function 
of both tissue interaction parameters as well as tissue morphology. The 
quantitative image, in principle, will not suffer from these defects: it would be 
independent of system settings and scanning procedure, and the image density 
will relate directly to an intrinsic tissue property uncomplicated by geometrical 
considerations. In this sense, the quantitative ultrasound image presents a new 
dimension of tissue information. However, quantitative imaging also demands 
elaborate data acquisition arrangements, as well as extensive computer-based 
manipulation of the acquired data. 

Two major approaches to quantitative imaging are being developed: transmission 
imaging and scatter imaging. Ultrasound computerized tomography, based on X- 
ray CT and utilizing the transmitted ultrasound waveform, has generated both 
attenuation and velocity maps of tissue (5-7). The velocity images are produced 
by inputting time-of-flight measurements into the reconstruction process; 
attenuation images are produced by utilizing measurements of the amplitude of 
the transmitted signal. Due to the presence of bone and air, these techniques are 
limited (perhaps fundamentally) to a few body sites such as the female breast or 
the neonatal head. Ultrasound computerized tomography is more subtle to 
implement than the X-ray case, since a number of simplifying approximations, 
such as straight-line propagation, isotrophy, and even scalarity of interaction, do 
not hold in all ultrasound situations. Measurement of the transmitted ultrasound 
field is complicated by the coherent nature of the generating and detecting devices 
commonly used. Despite these difficulties, reasonable quantitative attenuation 
and velocity breast images have been obtained in-vivo but it is as yet too early 
to assess their true diagnostic impact. 
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The second approach to quantitative imaging, which we term scatter imaging 
utilizes the scattered signal to produce quantitative maps of tissue interaction 
parameters. All scatter imaging methods (and many have been proposed) are 
related to classical inverse scattering techniques. Here, the problems are 
formidable and the methods have rarely been applied in-vivo. The proliferation of 
scatter imaging techniques may be conveniently classified by their data 
acquisition methods. Diffraction tomography (8,9) and its many variants 
(including filtered back-propagation) (10) is characterized by the measurement of 
the scattering into all angles. Impediography (11,12), including its variant, 
reflectively tomography (13), is characterized by the measurement of the 
backscattered signal only. Wave extrapolation imaging (14), utilizes what may 
be termed retroscattering, i.e. scattering into a relatively small cone about the 
strictly backscattered direction. This classification by data acquisition method is, 
in fact, dictated by the physics of the ultrasound/tissue interaction. Thus, each 
class of techniques will be expected to measure different parameters (or 
combinations of parameters) of the interaction, while variants of a given 
technique are merely different approaches towards mapping the same parameter. 

All quantitative scatter imaging methods are nothing more than specific 
realizations of special solutions to the inverse scattering problem. The first step 
in solving such a problem is the specification of an appropriate physical model 
and its associated wave equation. Next, a wave equation solution of the direct 
problem is obtained. This, in turn, leads to a specific description of the 
relationship between scattering parameters and measured quantities which, 
finally, must be inverted. The specification of the physical model is a crucial 
step in the procedure, since only those interaction parameters which are 
incorporated into the original wave equation can be reconstructed. However, 
tissue may not conform to the postulated model in all respects, and the measured 
data may well not be totally compatible with the original wave equation. Under 
these circumstances, the inversion scheme, which would be near-exact only under 
the hypothesis that the postulated physical model applies, produces an image 
which is, in some sense, incorrect, and thus hardly quantitative. We have used 
the term fuzzy (15,16) to describe an image which displays a particular parameter 
but is corrupted in an unpredictable manner by some other interaction 
characteristic not included in the original physical model.lt is apparent that all 
scatter imaging techniques are highly model dependent since the reconstruction 
algorithm is determined by the wave equation which characterizes the medium. 
Thus, if the model is incorrect, even infinitely good resolution and an exact 
reconstruction algorithm will not obviate a fuzzy image. Much attention has 
been devoted to problems of resolution and problems of distortion in ultrasonic 
imaging, while fuzziness has been virtually unappreciated as a problem (17). 
This unfortunate emphasis on blind computation in the absence of accurate 
tissue modelling produces images which may be highly resolved while being 
distorted and fuzzy. 
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The three problems of resolution, distortion, and fuzziness are, in fact, inter- 
related. Consider a conventional impediography inversion procedure. The three 
steps involved here are measurement of back-scattered signals, deconvolution to 
obtain the impulse response and inversion via an "impediography equation" 
which relates the impedance profile to the impulse response. The actual form of 
the impediography equation depends on the physical model assumed. Consider a 
constant velocity model in which frequency-independent attenuation (propagator) 
and impedance fluctuations only (reflector) are assumed. The final impediogram 
will be fuzzy if it turns out that backscattering from absorption fluctuations (not 
included in the inversion scheme) are significant. If the attenuation is, in fact, 
frequency dependent, then the ensuing pulse distortions will ensure an incorrect 
impulse response, which will, in turn, ensure a resolution loss in the displayed 
impedogram. However, we will nonetheless indicate the following scheme even 
bearing in mind that there may be some cross-links between the columns: 

FUZZINESS DISTORTION RESOLUTION 

I I ! 

REFLECTOR MODEL PROPAGATOR MODEL DATA MODEL 



It should be clear that physical modelling underpins the entire framework of 
quantitative imaging. 

An interesting example is afforded by diffraction tomography. Initially, the 
technique was based on the lossless Helmholtz equation (velocity fluctuations 
only). Given the measurement of angle scattering on which the technique is 
based, it is clear that the original technique would give rise to a distorted and 
fuzzy image. However, a much more detailed analysis by Mueller (18), has 
revealed that diffraction tomography, while not being fuzzy, displays an image of 
a rather unexpected combination of interaction parameters. An appreciation of the 
fuzziness concept runs deep in Mueller's work, even though the idea is not 
formulated as above, or analyzed as a general problem of quantitative imaging. 

Another interesting, and poorly addressed, problem associated with quantitative 
imaging is one we dub "freedom counting". The idea is easily explained by a 
simple example. Consider a two-dimensional velocity distribution, c(x,y). The 
velocity field shows two degrees of (spatial) freedom, x and y. It is clear that in 
order to reconstruct the velocity map, measurements with two degrees of freedom 
will also be needed. For example, in (two-dimensional) transmission 
tomography a time-of-flight projection (one degree of freedom) must be measured 
at all angles (second degree of freedom) in order to achieve a reconstruction; in 
(two-dimensional) diffraction tomography, the field scattered at all angles (one 
degree of freedom) must be measured for all angles of incidence (second degree of 
freedom). In order to measure both the attenuation, a(x,y), and velocity 
distribution, c(x,y), of a given (two-dimensional) structure, it is clear that two 
sets o f independent data, each with two degrees of freedom, will be required. The 
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simple expedient of freedom counting often allows a preliminary judgment as to 
whether an elaborate, and usually highly mathematical scheme can, in fact, 
reconstruct all the postulated interaction parameters within the context of the 
suggested data set 

Finally, to conclude, we believe that analytical and computational methods are 
available today to produce high resolution quantitative ultrasound images. 
However, the accuracy of such images are directly dependent on the exactness of 
the physical model assumed for tissue. In addition, the computations required to 
produce such images are quite extensive and will probably require the next 
generation of computer technology to be realistic. 
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1. Introduction 

There is a substantial component of diagnostic radiology (in fact, general clinical 
practice) where a diagnosis is either predicated upon a geometric determination 
or is ehanced by knowledge of some local geometry. Examples of this so pervade 
the literature that an exhaustive summary is unwarranted and inappropriate to 
a paper of this length, so we will content ourselves with one or two general 
examples. Measuring angle or length from projective modalities (plain film or 
US) is so widespread that one standard text (Keats and Lusted 1 ) is devoted to 
terse explications of standards and variations drawn from every major system 
in the body; general fetometry is routine in antenatal care, and newer examples 
appear in the literature frequently. Frequently, a geometry-related diagnosis 
can be achieved quite satisfactorily from a projective modality, and hence the 
necessity or motivation to utilize a 3D modality (such as MR or CT) may not 
be indicated. However, this is not always so; obvious indications for some form 
of 3-space geometry include guided biopsies (e.g., those conducted concomitant 
to mammography). Even here we concede that rudimentary 3-space geome- 
try (relative location, approximate size or volume) suffices in many cases, but 
when some form of fine-structure geometry is indicated, other (more rigorous) 
approaches are mandated. 

Say the task at hand involves the determination of a clinical result based 
on shape (e.g., the classification of structures into normal or abnormal). In our 
case, one of the authors (FDM) has hypothesized that there may be a corre- 
lation between the global geometry of the jugular foramen and communicating 
hydrocephalus secondary to achondroplasia. Our goal is therefore to provide 
some quantitative sense of 'normal' and 'abnormal* in the context of the jugu- 
lar foramen of the pediatric skull, and it is our contention that in so doing we 
must resort to the tools of differential geometry, algebraic topology and har- 
monic analysis on symmetric spaces. The space limitations of this paper dictate 
brevity of presentation, and so the remainder of the paper will concentrate on 
the provision of a skeletal outline of our approach. Future results in this area 
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(which we deferentially refer to as 'diagnostic geometry') will be more expansive 
both in the clinical applications and in the underlying mathematics. 

2. Materials and Principles 

The client clinical group comprise pediatric patients (achondroplasic and non- 
achondroplasic) who have been scheduled for an MR workup of the head. Typ- 
ically, 10 to 15 dual-echo (SD and T2) transverse images are obtained. These 
span a (typical) 20cm x 15cm Volume of Interest (VOI) which is presented to 
the imaging group as a multisignature lattice of (256 x 256 x m) (m in the 
range 10 to 15) voxels. Entities within the lattice are classified and either in- 
terpolated out to a continuous VOI (as is the case when the structure or entity 
is not amenable to a computed geometric representation, as in the determina- 
tion of total CSF volume), or they are converted to a differentiate structure 
of known topology for subsequent analysis. We concern ourselves here with the 
latter process. 

Conversion to a differentiable structure involves six basic steps: 

• Image Preprocessing (noise removal) 

• Cluster/ Classification of VOI 

• Conversion to Polyhedral Representation 

• Conversion to Parametric Representation 

• Conversion to Differentiable Structure 

Once the differentiable structure is known, shape analysis can proceed. Step 1 
(image preprocessing) is so common that we ignore it here. 

3. Entity Classification 

Since the image data set we acquire is multisignature, we opt for a controlled 
classification schema as the basic image interpretation method. The image I 
is simply some mapping from the VOI (i.e., some subset V C R 3 ) out to the 
(normalized) region [0, l] x [0, 1] c R 2 . Then the signature distribution of I is 
just the 2-space histogram of I (denoted Hj) } and clearly Hi : [0, lj X [0, 1] — ► 
[0, 1] when Hi has been normalized. The goal of the classification schema is 
to partition the domain of Hi (which we hereafter refer to as U) into disjoint 
subsets Ui whose union covers U . Then if / : p xyz — ► q G U s c U j Pxyz is said to 
be in class j. Notice that the primary goal of most classification algorithms is to 
partition the distribution space (U) of I (subject to some criterion of goodness), 
not the image space V itself (this being the real goal, but one secondary to the 
partitioning of U). For this reason, many classification algorithms perform in an 
unsatisfactory manner, in that the classes developed in V may not correspond 
to the structures one expects to find in V, For this reason, we use a controlled 
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classification schema, where the criteria of goodness are measured both in U and 
in V. The fundamental algorithm we use is a variant of the K-means 2 algorithm, 
which partitions U into convex subsets (under the I 2 metric) based on the notion 
of proximity of points in 17 to paradigmatic cluster centers ci, . . . , cm . Thus, 
if v xy is a point in the distribution space U of I, and ci t ... t CM are distinct 
points of U (called the class centers), then v xy is assigned to subset U n C U 
when d(v xyt c n ) is a minimum* (d(., .) is the I 2 metric; we will demonstrate in 
a later paper that the P metric is the only meaningful l p metric if convexity 
is an issue). The algorithm iteratively recomputes the c», terminating either 
when a convergence criterion is met, or when an iteration limit is exceeded. 
Clearly, algorithms such as this are highly dependent on the choice of initial 
class centers. Our control schema uses a data-pruning technique coupled with 
a relaxed crossover between the algorithm's performance over both U and V 
(e.g., the schema monitors the behaviour of edges in V under partitions of U) 
to mitigate the effects of a bad initial choice of class centers. 

4. Polyhedralization 

The data structures obtained after classification include a parallel image set 
C : V C R 3 -> {0, 1, . . . , M - 1} whose 'voxels' c xyz identify the class (0 to 
M-l) to which voxel v xyz €E V was assigned. Interclass boundaries are then 
developed from C, and those which are adjudged to belong (adhere in our ter- 
minology) to a target structure (e.g., a foramen) are retained. Clearly, this step 
is one of the most complex in the entire process, and we concede that at this 
stage our inference engine is populated with ad hoc rules which are 'demon- 
strated to work'. Ideally, we would like to construct an inference engine whose 
ruleset is more globally meaningful. One of our principle goals is the identifica- 
tion of 'anchor processes' (i.e., anatomical structures or physiologic byproducts 
whose geometry and signature we believe can be identified with high probabil- 
ity; in fact, our initial success in this area came from pelvic imagery 3,4 ). From 
this point in the process onwards, the original image data I can be discarded; 
no further knowledge of the MR signatures is needed. Voxels v xyz which lie on 
the boundary of a target structure, are of constant z (i.e., all lie in the same 
image plane) and which lie on a simple closed connected curve are called contour 
voxels. Although there may be some merit in attempting to produce a differ- 
entiate surface directly from these contour voxels, we have adopted a bipartite 
polyhedralization process which generates two (not necessarily redundant) in- 
termediate structures. Our first goal is to throw a polyhedral surface between 
the contours. 

Much has been written on this (e.g., see 6,6 ), and although this may seem 
like a solved problem, it transpires that this is not the case. Essentially, the 
problem reduces to ensuring topological fidelity between the original surface 
and the polyhedral representation of that surface given the contours from the 
image. Virtually any of the standard algorithms can be demonstrated to fail 
under pathological (and often not-so-pathological) circumstances. Those based 
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on Voronoi tesselations may tesselate over holes, while those based on trian- 
gulations may develop self-intersecting surfaces. It seems apparent to us that 
algorithms which are to tesselate surfaces correctly must have some a priori 
knowledge of the topology of the surface they are working over (e.g., at least 
the fundamental group of the surface), but it is also clear that local knowl- 
edge of a topology (developed say as the algorithm builds the surface) may not 
suffice to inhibit the construction of a toplogically invalid surface. Our cur- 
rent approach to this problem involves specifying the topology of the surface 
to the reconstruction algorithm ab initio (and for reasons to be seen later, we 
restrict our attention to surfaces homoemorphic to the sphere) and employing 
a symmetry-reduced expansion of the classical Fuchs 5 triangulation strategy. 

The Fuchs' strategy begins by building a torroidal graph G whose vertices 
gij correspond to an arc-related pairing between a point pi on one contour 
and qj on the other. (The surface is built up by unioning subsurfaces con- 
structed pairwise between adjacent contours). Arcs in G are directed, and may 
either connect (ft,/, &+i t y) (in which case they correspond to a triangle with 
components < (pi,Pi+i), (ft+n4j)> ($/,Pi) >) or (#.y,#,y+i) with obvious cor- 
responding triangle. Although Fuchs gives a 'safe strategy 9 for constructing a 
surface based on the development of a subgraph in G which sstisfies three prop- 
erties, it transpires that these properties are too weak to guarantee topological 
fidelity. There are in fact some ^Tm^' potential acceptable surfaces (subgraphs) 
when the two contours have n and m contour points (respectively), only some of 
which are topologically valid (i.e., do not have self-intersections). As indicated 
above, no adequate local graph property can exclude these topologically invalid 
surfaces. However, one can show that many of these subgraphs are related un- 
der the action of permutation groups to each other (are in the same equivalence 
class) , so that the number of candidate subgraphs one has to develop can be 
reduced to locating the coset leader subgraphs using the Polya Enumeration 
Theorem. When this has been done, each candidate subgraph can be expanded 
quickly in a parallel environment (we use a 4-processor transputer network) and 
tested for topological integrity. Only those which are topologically valid are re- 
tained for further consideration. In fact, we conjecture that a surprisingly low 
percentage of the total number of 'acceptable surfaces' are in fact acceptable 
topologically. 

At the end of this procedure, we have a surface which is topologically valid 
(in the fact that it at least is not self- intersecting) composed of the union of 
triangular (planar) patches. Unfortunately, the conversion from a triangulation 
to a parametric representation is computationally too expensive, so we develop 
a secondary polyhedral structure which is constructed from the union of quadri- 
lateral facets (and we accept the potential for a severe loss of planarity here). 
That the triangulation should proceed the quadrilateralization should be obvi- 
ous; however, the triangulation is used in all visualization algorithms we employ, 
so while its validity as a geometrical structure is of questionable value to us, it is 
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of import in a secondary fashion. The quadrilateral structure is pairwise regular, 
in that it contains the same number of polygons between each contour. It is 
this structure which is used as a basis for the development of the differentiable 
structure. An example of such a structure is given in Figure 1; this happens to 
be the inferior surface of a urinary bladder; note that the apparent elongation 
is a display artifact. 

5. Differentiable Structure 

The pairwise regular polyhedron developed in the last section can be converted 
to a differentiable structure by associating a parametric patch (say a bicubic) 
with each component polygon. Any one of a number of standard approaches 
can be employed to accomplish this, but we impose the additional criteria that 
the surface so developed be fair with respect to the polyhedron (i.e., introduce 
no concavities when the data does not imply a concavity - see 7 ) and be at least 
C 2 , since we will want to compute derivatives up to at least second partials. 
The union of these parametric patches constitutes a compact 2-manifold M, and 
we speak of the functions g : P C M — * (0, 1) x (0, 1) taking open sets on M to 
open sets in R 2 as the atlas on M. Since our primary reason for developing this 
differentiable structure is to quantify shape, further results on the differentiable 
structures are best developed in that context. 

6. Shape 

For meaningful comparative purposes, shape is clearly best described in terms 
of some numerical quantity (or set of numerical quantities). We would like our 
shape descriptors to be a) tunably unique to a given manifold; b) independent of 
the atlas describing the manifold; and c) invariant under certain rigid motions. 
By "tunably unique" to the manifold we mean that the descriptors should be ef- 
fectively absolutely unique to the manifold (by their definition), but constructed 
in such a fashion as to allow easy blurring between two essentially equivalent 
manifolds (the notion of harmonic series expansion should be apparent here). 
We also clearly require invariance under certain rigid motions; that a shape de- 
scriptor be incapable of identifying the same entity simply because that entity 
has rotated (even slightly) is clearly untenable. We can clearly eliminate the 
effects of translation of the manifold by requiring that the shape decriptors be 
a function of some rigid object and providing a mapping between the actual 
manifold under consideration and the rigid object. In fact, this drops out nicely 
from the harmonic analysis. 

It is clear from the preceeding discussion that we can restrict our attention 
to formulating a shape descriptor function invariant under the action of SO (3) - 
the group of rotations in 3-space. The homogeneous space for this group is the 
sphere S 2 (which is left invariant under the action of SO(3)), and it transpires 
that the harmonic analysis over S 2 with respect to the representation of SO (3) 
on the vector space of infinitely differentiable functions on S 2 gives rise to a 
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shape descriptor function of the form 



where / is some meaningful shape function (such as Gaussian curvature, mean 
curvature or geodesic curvature) and the Y} n are Associated Legendre functions. 
It transpires that such a formulation exhibits all of the "nice" properties stated 
above 8 , and we then call the set {A l m } the Spherical Shape Descriptors of M. 
For M to have such a set of shape descriptors, it is clear that M must be related 
to S 2 (i.e., we must be able to construct a differentiable mapping between M 
and S 2 ), and this brings us to the central point of this paper, which we state 
without proof: 

Theorem: Let M be a compact 2-manifold constructed as above, and let K be 
the Gaussian curvature on M. Then provided the integral of the total curvature 
is as below 



then there exists a diffeomorphism between M and S 2 and M has a rotationally- 
invariant shape descriptor function as above. 

Proof: This is just the global Gauss-Bonnet theorem followed by the construc- 
tion of a differentiable mapping between the parameter space of M and the 
parameter space of S 2 . 



The authors wish to express their gratitude to the Imaging and Graphics Labo- 
ratory (Booth Research Center) for providing support, and to the Department of 
Radiology (University of Connecticut Health Center) for assistance in acquiring 
the MR data. 



(1) : T.E. Keats and L.B. Lusted: "Atlas of Roentgenographs Measure- 
ment", Year Book Medical Publishers, Chicago, 1985 

(2) : J.T. Tou and R.C. Gonzalez: "Pattern Recognition Principles", Addison- 
Wesley, Reading MA 1974 

(3) : I.R. Greenshields and K. Yoon: Projecting MRI Data into Parametric 
Surfaces, Proc. SPIE Vol. 1092 pp 54-59, 1989 

(4) : LR. Greenshields and K. Yoon: Determination of and Visualization 
of Anatomical Structures from Magnetic Resonance Imagery, Proc. 
WOCON-89, pp. 478-479, Korea Information Society, Seoul, Korea, 1989 

(5) : H. Fuchs et. al: Optimal Surface Reconstruction from Planar Con- 
tours, C.ACM Vol. 20 No. 10 pp. 693-702, October 1977 




7. Acknowledgments 



9. Literature Cited 



743 



(6) : S.B. Xu and W.X. Lu: Surface Reconstruction of 3D Objects in 
Computerized Tomography, Computer Vision, Graphics and Image Pro- 
cessing 44, pp. 270-278, 1988 

(7) : T. Rando: "Automated Fairness in Computer Aided Geometric 
Design", Ph.D. Thesis, Dept. of Computer Science and Engineering, Univ. of 
Connecticut, 1990 

(8) ; I.R. Greenshields and C.J. Piwang: Some Aspects of a Three Dimen- 
sional Shape Descriptor Function, Proc. SPIE Vol. 937 pp. 303-309, 1988 



Figure 1: Inferior Surface of Bladder 



A User-Friendly Human-Computer Interface for a 
NMR Workstation 

Gabriele von Voigt 

Department of Computer Science, University College London, 
London, UK 



Abstract 

Every computer-assisted workstation in a hospital has its own user 
interface. As a result, the medical staff who operate these machines have 
to learn a variety of systems, which are often not easy to handle and not at 
all self-explanatory. It is quite likely and understandable that the user 
has problems in operating these systems and thus mistakes are made. 
Would it not be advantageous to learn to handle only one user interface 
which is easy and quick to use? 

The purpose of this paper is to show criteria for a user-friendly human- 
computer interface (HCI) for a NMR workstation and workstations from 
other scanner types. 

The first part of this paper contains the definition of the users and an 
analysis of the benefits and costs of introducing the system. In the second 
part, one finds the user and task characteristics for a picture-oriented 
medical workstation, which are a consequence of a field study which 
examined a limited number of NMR workstations. In the third section, the 
author shows the derived characteristics of a general purpose HCI (GP- 
HCI) for medical workstations (MWS). The fourth section outlines an 
experimental implementation of such an interface and, in the last section, 
examples of specifications for tests regarding the user-friendliness are 
described. 



Introduction 

The main goal is to design a user-fiiendly system. This means, the system 
should be easy to operate, provide suitable functions to assist the user 
fulfilling his task and should be enjoyable to use. Thus the design has to 
be matched to the user. The presented process is based on the idea of 
user-centred design (HUSAT, 1989). 
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Section I 

LI. Understand the User 

To match the design to the user, we are in an unfortunate situation, that 
the user does not exist! One suggested solution to this occurring problem 
is to build user-groups. These groups are formed depending on the tasks 
of the users. We can form the following seven user-groups: 

1. Referring Physicians and Specialists such as Surgeons 

2. Radiologists and Radiographers 

3. Nurses 

4. Administrators 

5. Managers 

6. Department of Health and Social Security (DHSS) Personnel 

7. Patients 



L2. The Analysis of the Goals-, Benefits- and Costs 

The analysis states the goals of the tasks for the user-groups, their benefits 
from this MWS and their costs of using or choosing this MWS. Only a 
system with copious benefits should be developed. 



Section II 

II.1. User Characteristics 

The characteristics of each user-group has to be determined. These 
characteristics can lead directly to product requirements which we have to 
consider. In this paper the concentration will be on the direct user groups: 
radiologists (including radiographer) and physicians (including other 
specialists). The study which was carried out showed, for example, the 
following characteristics for the user-group of radiologists: 

Radiologists have the following characteristics: 

* low -> medium medical knowledge 

* high intellectual ability 

* high radiological knowledge 

* low -> medium computer knowledge 

* low -> medium manual keyboard skills 

Also, their work is repetitive and requires very frequent use of the 
system. 
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II.2.User Tasks 

In the first step, the task environment and the working conditions have to 
be examined. Depending on the direct user-groups, there are mainly two 
areas in which the NMR workstation will be placed: one is the NMR unit 
based in the radiological department, which will provide adequate 
workspace, satisfactory lighting and it will be positioned in the 
surroundings with a strong magnetic field. The other environment is the 
ward where the workspace is normally limited, the lighting can be 
variable and it could be noisy. 

The following product requirements can be derived: 

- the size of the workstation should be small 

- the brightness and contrast must be adjustable 

- audible output should be avoided. 

The working conditions on the ward and in the NMR unit are quite 
different, in the NMR unit, only a limited number of people work together 
with the MWS whilst creating the NMR scans. The radiologist normally 
handles the system and one or two other clinicians overlook it. 

On the ward and especially during the presentation of the patient's record 
to colleagues more than one person needs to see the contents of the 
display of the MWS. On the other hand, the ward is a public place but the 
patient's data must be treated as confidential. Also, it is possible that the 
physician or surgeon is interrupted by a more urgent task during his work 
with the computer. 

We can conclude the following product requirements: 

- the screen must be big enough to allow for two or three 
users simultaneously 

- the MWS must contain facilities for the visualization of 
the screen to multiple users 

- the display needs a viewing angle limitation and a "blank 
screen" function 

- the system needs quick store and retrieval functions 



Section III 

Characteristics of a general purpose HCI 

1. Attractive to new users: The MWS should be a communication 
medium for the radiologist user group and the physician user-group. 
Thus, new users have to be encouraged to use the system. 
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2. Self-explanatory and easy to learn: The physicians are novices and 
they will not be interested in going through a long tutorial or in working 
through a handbook. 

3. Easy to use including compatibility with existing system: It is easier to 
extend existing knowledge than to learn a totally new concept. Thus the 
present knowledge base should be utilized. 

4. Low error rate and able to be used rapidly: The clinician will feel 
more confident in using the MWS if he makes no errors during the 
interaction. Also, every recovery of an error takes time, but in a hospital 
the time which he can spend with the computer is limited. 

5. lnterruptable and easy to resume after an interruption: Earlier, the 
task analysis showed that the physician might be interrupted during his 
work with the MWS. Afterwards he will want to restart at the point 
where he stopped. 

6. Relieving the user's memory load: Interaction sequences and 
previously worked out commands should not be required again by the 
physician or radiologist. 

7. Laid out consistently: The necessary interaction should be easy to 
remember. To achieve that end, the clinician must be able to create a 
mental model of the system. Therefore, one philosophy should guide the 
physician or radiologist through the interactions, even if both user-groups 
are quite different. 

8. Providing quick response to frequent users and providing prompts to 
remind infrequent users: Whereas the radiologists are frequent users, the 
physicians are infrequent users. 

9. Supplying information as requested and not insulting the intelligence: 
The designer also has to pay attention to the intelligence of both user- 
groups. 

10. Supportive in the search for a particular patient folder: Both user- 
groups have in common that one of their basic and frequently used 
functions is the storing and retrieval of a particular patient folder. 

11. Able to present different patient data in a uniform way: They are 
familiar with the concept of a patient folder and to the information being 
presented in a uniform way. A fever curve has to be displayed as a curve, 
if this is the usual method, and not as an array of measurements. 
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12. Able to display 3D image data: All natural objects are three 
dimensional. Thus we are used to understand an object in three 
dimensions. It seems obvious to the author that patient data which 
contains 3D information, like NMR scans, should be displayed as a three 
dimensional object as well as in two dimensions. 

13. Supportive in interacting with the 3D image data: That requires 
interaction devices which allow the handling, the operating with and the 
manipulation of three dimensional information. 

14. Using highlighting and pseudo colour for important items: The 
patient data, together with the necessary interaction information presents 
a great amount of information which has to be displayed. Therefore, a 
good use of colour can help to give an emphasis on special objects. 

15. Using the human sensibility of motion: Also, the human eye is 
sensible to motion. A good designer will use these and other ergonomics 
for his layout. 

16. Indicating the needed time for the execution of an operation: If a 
computer function needs some time to calculate the results, it is essential 
that the user gets an estimate of the execution time. 

17. Indicating the state of the system and supportive in the orientation 
throughout the system: The MWS should make it always clear, in which 
state it is. The feedback is necessary for the clinician to judge if the 
computer action fits his expectations. It is important that he never loses 
his orientation whilst working with the system 

18. High reliability: System crashes are not permissible. 

19. Secure - as required by the Data Protection Act: In the hospital, we 
have constraints due to the Data Protection Act. Therefore, during the 
interaction, we have to prevent anyone overlooking the physician's work 
and the MWS must be secure against unauthorized access. 

20. Challenging and enjoyable to use: Last but not least, the use of the 
MWS should be challenging and enjoyable. 



Section IV 

IV.L Design of the GP-HCI 

This section outlines a possible design of a general purpose HCI for a 
MWS regarding the two direct user-groups. There are mainly three areas 
of interest, which will be explained in the following. 
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IV.2. Style of Interaction 

The suggested design has the four most important features: 

* user-guided dialogue 

* direct manipulation 

* flexible menus and icons 

* pre-stored values 

The dialogue should be user-guided to give the clinician more control 
and flexibility. If an entry for a new patient in the database has to be 
made, the entering of the data should be done by using form-filling 
dialogue method. All other data entry will be supported by direct 
manipulation. The user's commands, decisions and option choosing will 
be supported by flexible menus. This assists the human hierarchical 
memory structure and process of choice. If the choices can be shown 
graphically, the use of icons should be considered. For frequent inquiries, 
pre-stored values should be available. These can be defined as macros 
with their own icons. 



IV .3. Appearance 

The two following features characterize the design: 

* multiple windows 

* graphics and alpha-numerical data 

The screen is laid out by using fixed and overlapping windows which are 
under user control. A fixed menu in the top of the screen shows all main 
choices of the MWS continually. In this way the major part of the screen 
is available for displaying the relevant data at any time, and the user still 
has an overview of the main features of the whole system. The additional 
options relating to the main choices are hidden in pull down menus, thus 
the screen remains uncluttered. Graphical and alpha-numerical data can 
be visualized. The most relevant data should always be presented in the 
middle of the screen. 



IV.4. Input 

The input will be assisted by the following: 

* function keys, 2D and 3D mouse 

* help button 

Function keys can be used if the interaction is rapid and the information 
is simple or context free. The 2D mouse is suitable for the flexible menu 
approach and the 3D mouse stands for all appropriate 3D input devices 
which allow the easy handling of 3D patient data. Context sensitive help 
should be always available by using the help button. 
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Section V 

Test specification 

Having a prototype, tests have to be done which show if the 
characteristics outlined in section III are fulfilled. As an example, the 
specification of the tests for two of the characteristics will be shown. 

Test 1 shows if the MWS is self-explanatory: 

Test procedure: laboratory test with and without tutorial 

Measure: time to complete given tasks 

data collection: observation, on line timing, questionnaire 

One group of users uses the MWS prototype with and another group 

without a tutorial or handbook. We measure the time they need to 

complete several set tasks. We should also observe how they handle the 

MWS and ask them afterwards about their difficulties. 

Test 2 can tell us if the use of the MWS is challenging and enjoyable: 
Test procedure: prototype in area for casual use 
Measure: count users, collect their comments 
data collection: data record, questionnaire 

We place a prototype in an area which is accessible for casual users. We 
count how many users approach the MWS, what they do and how many 
tasks they solve and we ask them about their criticisms. To achieve this, 
we create automatically a data record and let the users fill in a 
questionnaire. 



Conclusion 

The user interface plays a very important role for the acceptance of a 
medical workstation. The author suggests to facilitate the idea of user- 
centred design for the design process. After characterizing the users and 
tasks, product requirements can be derived. Then a prototype can be built 
and tested. These tests may lead to re-design of parts or in the whole as 
appropriate. 

This paper can only present some of the necessary requirements, but 
could help other interested parties. The design of a GP-HCI in this area is 
a great challenge and, in the eyes of the author, a step in the right 
direction. 
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INTRODUCTION 



Gestational age (GA) and fetal weight 
(FW) are often estimated by comparing sono- 
graphic measurements of the fetal body and 
head to standard values in published tables 
or charts. Ideally, the standards used to 
analyze a given patient's fetal measurements 
should be derived from the same population as 
that of the patient. Unfortunately, the 
"standard" charts and equations commonly used 
are often based upon a different population 
that may exhibit a quite dissimilar fetal 
growth pattern; calculations of both GA and 
FW could therefore be in serious error. 

Developing a fetal growth standard from 
one's own patient population is more advan- 
tageous, but requires the manipulation and 
intricate analysis of large volumes of nu- 
merical data. Such a "population-specific" 
reference can be developed, however, using 
inexpensive software with an IBM-compatible 
personal computer (PC) . The PC can also be 
used to analyze trends of fetal growth within 
a particular population. 

MATERIALS AND METHODS 

Patient data is recorded into a spread- 
sheet ("Quattro", Borland International, 
Scotts Valley, CA) . A non-relational data- 
base ("Q&A", Symantec, Cupertino, CA) 
facilitates patient follow-up. IBM-com- 
patible PC's with a 80286-based central 
processing unit, a 20 MB hard drive, and 
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either 512KB or 1024KB of random access 
memory are used. Both Quattro and Q&A include 
routines by which data can be easily imported 
or exported between the two. 

Data acquisition involves several steps: 

1. A Quattro spreadsheet file is 
designed with each row representing a data- 
base record for a single ultrasonic exami- 
nation on a given patient. Each spreadsheet 
column represents a data-base field (Patient 
Name, Date of Exam, etc.). A corresponding 
Q&A data base file including several of these 
same fields is also designed. 

2. At the time of the sonogram, the 
patient's name, age, hospital chart number, 
date of the sonogram ("Date of Exam") , fetal 
biparietal diameter (BPD) , head circumference 
(HC) , abdominal circumference (AC) , and femur 
length (FL) are each entered in a separate 
spreadsheet cell. Menstrual age (MA) at the 
time of examination ("MA by Exam") is es- 
timated according to the method of Hadlock et 
al 1 and is entered in another cell. The 
expected date of confinement is calculated 
and entered into still another cell ("EDC by 
Exam") using the formula: 

(Date of Exam) - (MA by Exam X 7) + 280. 

3. Each new line in the spreadsheet is 
then imported into a data base record in a 
Q&A file. Additional Boolean fields in each 
Q&A record indicate whether the mother's 
delivery date has been established ("Delivery 
Follow-Up?" ) and whether the baby's hospital 
chart has been reviewed ("Baby Follow-Up?") ; 
"No" is initially entered in both of these. 

A character field is reserved for the baby's 
hospital chart number. 

4. At regular intervals thereafter, the 
Q&A data base is searched for those mothers 
who should have delivered by the time of re- 
view. Records are retrieved by utilizing 
"EDC by Exam,<= (selected date)" and 
"Delivery Follow-Up?" = "No" as search cri- 
teria. A columnated report is then printed 
listing the patient names and hospital 
numbers for these retrieved records. This 
list is then given to medical records 
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personnel so that the mothers' medical charts 
can be obtained. 

5. The mother's chart is reviewed. 
Patients with twins or with such conditions 
as chronic hypertension are deleted. The 
actual date of delivery for acceptable 
patients is recorded in the appropriate 
spreadsheet row. The baby's hospital number 
is obtained either from the mother's hospital 
chart or from neonatal nursery records. If 
all these data are available, then the baby's 
hospital number is entered into the appro- 
priate field of the Q&A data base record and 
"Yes" is entered in the field "Delivery 

Fol low-Up?". 

6. The Q&A data base is then searched 
for those mothers whose birth records have 
been reviewed but whose babies' records have 
not. These records are retrieved by using the 
search criteria "Delivery Fol low-Up?" = "Yes" 
and "Baby Follow-Up?" = "No". A second col- 
umnated report form is printed with the names 
of the mothers and the chart numbers of the 
respective infants. This list is then given 
to medical records personnel and the infants' 
hospital records are obtained and reviewed. 
Infants with low Apgar scores (less than 7 at 
five minutes) , prematurity , low birth weight, 
etc., are first deleted. Weight, crown-heel 
length, and head circumference at birth are 
then recorded. "Actual gestational age at 
birth" is estimated by a modified Dubowitz 
score 2 . If a specific Dubowitz score is not 
listed but the infant is felt to be "term", 
an "actual gestational age" of 40 weeks is 
assigned. 

7. The above data are all entered into 
the respective cells on the appropriate row 
of the original Quattro spreadsheet. Actual 
menstrual age at the time of initial sono- 
graphic examination ("Actual MA at Exam") is 
then retrospectively calculated using the 
formula: 



Actual MA at Exam 

= (Actual gestational age at birth) - 
{ (Actual Date of Delivery) 
- (Date of Exam)} / 7. 
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The completed spreadsheet rows are then moved 
into another identically designed spreadsheet 
file entitled "Complete". 

RESULTS 

The "Complete" spreadsheet file pres- 
ently includes the data from approximately 
600 obstetric examinations performed at the 
Louisiana State Medical Center in Shreveport. 
This "Complete" file can be sorted and re- 
edited to produce a spreadsheet in which the 
column "Actual MA at Exam" represents the 
independent variable and the column for each 
fetal parameter (BPD, HC, AC, or FL) rep- 
resents a dependent variable. In this way, 
one can produce graphs (Fig. 1) or tables 
representing a standard reference for a 
population's fetal growth during gestation. 
Data from other institutions can be imported 
into and graphed from the same spreadsheet so 
that different patient populations can be 
compared (Fig. 1) . 

The spreadsheet data can also be used to 
determine statistically valid and population- 
specific regression equations for directly 
estimating gestational age at the time of a 
sonogram from fetal measurements. To this 
end, BPD, HC, AC, or FL are considered as 
independent variables and MA as a dependent 
variable. Establishing such regression for- 
mulae is greatly facilitated by saving the 
spreadsheet file in a "Lotus-compatible", 
ASCII or comparable standard format so that 
data can be readily exported to other 
statistical or graphics programs. Our re- 
gression formulae will be the subject of a 
future report. 

One can also analyze errors in gesta- 
tional age determination. In this case, a 
new column entitled "Error" is inserted. 
Values in this column are determined by the 
formula: 

(Actual MA at Exam) - (MA by Exam) . 

This value can be then plotted as a function 
of Actual MA (Fig. 2). 
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Figure 1. Growth of abdominal circumference (AC) during 
gestation in LSU-Shreveport patients compared 
to mean AC in those studied by Hadlock et 
al 1 . LSU patients demonstrate a relative decel- 
eration of growth after approximately 32 weeks. 
Graphs for BPD, HC, and FL showed similar 
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Figure 2. Menstrual age (MA) error (mean +/- 2 SD) as 
a function of duration of pregnancy. Error = 
(MA predicted by sonogram from Hadlock stan- 
dard 1 ) - (actual MA at time of ultrasound as 
computed from post-natal gestational age). 
Note increasingly negative values during the 
third trimester. 
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DISCUSSION 

A "standard" gestational age chart 
derived from one patient population may not 
be applicable another because of differences 
between the two groups' fetal growth rates. 
Our patients' BPD, HC, FL, and particularly 
their AC did tend to be smaller, particularly 
after 32 weeks, than those of the population 
used by Hadlock et al 1 to derive one of the 
more widely used standards (Fig. 1) . As a 
result, our patients' MA during the later 
stages of pregnancy was underestimated by up 
to 2-3 weeks (Fig. 2) . This discrepancy can 
have obvious implications for patients with 
such conditions as diabetes mellitus or 
intrauterine growth retardation in whom ac- 
curately timed early delivery may be neces- 
sary to enhance infant survival. The PC 
approach described herein can, however, pro- 
vide a relatively simple way to define a 
fetal growth chart that is more apropos for 
one's own clinic. The PC can also help test 
this new population-specific standard; for 
example, another spreadsheet can be designed 
where the expected date of confinement pre- 
dicted by the new chart is compared with the 
actual date of delivery. 

It is conceivable that the apparent 
differences between our population and that 
of Hadlock et al 1 are due more to dissimilar 
methodology than to dissimilar patient popu- 
lations. The "gold standard" of gestational 
age for most fetal growth references, in- 
cluding that of Hadlock et al 1 , is defined 
prospectively by the first day of the pa- 
tient's last normal menstrual period (IMP) , 
while our "Actual MA at Exam" is computed 
retrospectively from the infant's gestational 
age at birth. Both methods may still provide 
similar results 3 ; moreover, the retrospective 
method, unlike the prospective one, elimi- 
nates the bias produced by excluding women 
with uncertain or unreliable dates 4 . 
Nevertheless, these methodological dis- 
crepancies do not in themselves detract from 
the use of a PC to evaluate fetal growth 



757 



trends. A spreadsheet file can be designed 
in which "Date of LMP" can be entered in a 
date cell; time since that date can then be 
easily computed and placed in another cell or 
column (e.cf., "Menstrual Age from IMP") and 
then quantitatively compared with the "Actual 
MA at Exam" determined retrospectively from 
Dubowitz scores. 

The PC is also sufficiently flexible to 
support undertakings beyond the derivation of 
a population-specific fetal growth chart. Any 
variable or quantity that can be sorted or 
searched in a database or calculated in a 
spreadsheet can be investigated. For 
example, maternal parity, weight, or smoking 
history can be recorded and correlated with 
changes in BPD, etc., during gestation. Data 
can be exchanged between different computers, 
researchers, and programs. Additionally, 
data files can be easily transmitted via 
modem to colleagues in distant cities. The 
procedures described in this paper serve only 
as a suggestion; the actual possibilities and 
applications of the PC are limited only by 
one's imagination. 
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ABSTRACT 

Average glandular tissue dose in mammography is generally 
determined from published tables with knowledge of the breast entrance 
skin exposure, beam quality (HVL), breast thickness and breast 
composition. Utilizing a carefully benchmarked Monte Carlo simulation we 
have found that, for commonly employed molybdenum target/molybdenum 
filter source assemblies, average glandular dose also depends on x-ray tube 
voltage. For the same HVL errors of 10% or more possible in estimating 
average glandular dose if the x-ray spectrum employed is different from the 
one used to develop the reference dose table. 

INTRODUCTION 

Established computer applications in radiology include radiation 
therapy treatment planning, billing, CT and radiology information 
management, and MRI. More recently PACS and teleradiology, Computed 
Radiography, and 2D and 3D image processing have been introduced. In 
addition, there are a number of other less visible but fundamental 
applications. Monte Carlo simulation and modeling of the x-ray interactions 
in tissue is one of such example. In this paper, a Monte Carlo simulation of 
photon transport in breast tissue and the associated absorbed dose is 
presented. 

At present, mammography is the most accurate and reliable means of 
detecting minimal, npn-palpable breast cancers' 1 ' 2 !. However, the 
carcinogenic risk associated with the absorbed radiation dose to the breast, 
and in particular glandular breast tissue, is of concern and mammography 
doses are routinely monitored^ 3 '. If the normalized average glandular dose 
D gN (the average glandular dose per unit entrance skin exposure) is known, 
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the average glandular dose D g can be computed from the product of D gN 
and the breast entrance skin exposure X ESE . That is, 

D g = Dc^XESE (1) 

where the respective units of D gl X ESE and D gN are rad, roentgen (R) and 
rad/R. 

The evaluation of the glandular dose delivered in a mammography 
exam is thus reduced to a measurement of the breast Entrance Skin 
Exposure (ESE) and an evaluation of the normalized average glandular 
dose D gN , which is generally assumed to be a function of beam quality 
(HVL), x-ray tube target material, breast thickness and breast composition. 
Utilizing these assumptions D gN has been studied experimentally^ 4,5 ! and 

with Monte Carlo simulations^ 6,7 !. Based on these studies, the U.S. Food 
and Drug Administration and National Council on Radiation Protection and 
Measurements have published tables of D gN as a function of beam HVL, x- 

ray tube target material, breast thickness and breast composition^ 7,8 !. 
These tables assume that the incident x-ray spectrum is adequately 
characterized by HVL and x-ray tube target material. Additional factors such 
as x-ray tube voltage, voltage waveform and beam filtration which are 
necessary to properly characterize an x-ray spectrum are not addressed. In 
this paper we show that tube potential is an important consideration. 

METHODOLOGY 

X-ray photon transport is a stochastic phenomenon, and Monte Carlo 
techniques simulate this process^ 9 !. By using random numbers, i.e., by 
"throwing a dice M f this technique performs sampling experiments on 
photon-tissue interactions by tracing the histories of individual photons. 
Macroscopic behavior is obtained by averaging the results of repeated 
experiments. For example, if Dj is the dose delivered by the ith photon to 
tissue and N photon histories are traced, then the average dose per photon 
is 

D ave = (ZDi)/N (2) 

Since each X-ray photon interaction in tissue is independent, the 
central limit theorem guarantees that the dose thus found will be a good 
approximation of the true dose as long as the total number of photon 
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histories traced is sufficiently large. In our simulation 1 million photon 
histories were traced per run, and the relative error of D ave is less than 1%. 

In mammography there are three physical processes of interest — 
photoelectric absorption, incoherent scattering and coherent scattering. 
The quality of a Monte Carlo simulation also depends on the accuracy with 
which these processes are described. Care was taken in this step. 
Interaction sites were sampled by using the total attenuation coefficients of 
breast tissue, and the interaction types were sampled by using the photon 
cross section data for the three physical processes involved in 
mammography. For coherent and incoherent scattering electron-binding 
effects were taken into account by employing the appropriate atomic form 
factor (coherent) and scattering function (incoherent). The photon cross 
sections, form factors and scattering functions employed were obtained 
from the Evaluated Nuclear Data File (ENDF)t 1 0 h 

The details of Monte Carlo simulations of x-ray photon transport have 
been described in the literature^ 9 !. Our simulations utilized a modified 
MCNP code! 1 1 I. Modifications were made to adapt the source code so that 
it runs under the UNICOS operating system (it was written originally to run 
under COS operating system of Cray supercomputers). Other source code 
modifications were made so that the primary and scatter interactions could 
be monitored, since this information is of interest in mammography. Also, in 
order to use the MCNP package to determine normalized glandular breast 
dose, command files were designed so that the photon energy, geometry, 
photon cross sections and energy deposition were appropriately treated to 
simulate the problem of interest. 

The breast was assumed to have a semi-elliptical cylindrical shape 
(semi-axes 9 cm and 8 cm) with a uniformly distributed mixture of glandular 
tissue and adipose tissue contained within a 0.4 cm thick skin layer. The 
chest wall dimension was 18 cm and the chest wall-to-nipple dimension 8 
cm. This shape is representative of a typical compressed breast for the 
craniocaudal projection and is the same as that employed to develop the 
FDA's dosimetry tables^. In the simulation, a range of breast compositions 
and breast thicknesses were studied. The elemental composition of the 
glandular and adipose tissues were taken from the measured results of 
Hammerstein, et atf 4 !. 

Normalized average glandular doses were computed as described 
above at 28 energies ranging from 7.5 kev to 34.5 kev with a energy 
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resolution of 1 keV. Polychromatic normalized average glandular doses 
were calculated by appropriately weighting the monoenergetic results. The 
x-ray spectra used in the computation were obtained employing the 
constant potential molybdenum target, molybdenum filter x-ray spectra 

model of Tucker, Barnes and and ranged in x-ray tube potential from 

25 kVp to 35 kVp. At a given tube potential different HVLs were obtained 
by varying the thickness of polycarbonate plastic (i.e., compression paddle 
thickness) placed in the beam. 

Prior to computing the average glandular doses, a number of checks 
were run and the predictions of our simulation were compared with the 32 
kVp scatter-to-primary ratio measurements of Barnes and Brezovichf 1 3 1 and 
depth dose measurements of Hammerstein, et alt 4 '. 

Table 1 . Comparison of measured and computed scatter-to-primary ratio for 
6 cm thick circular Lucite phantom 



Field Scatter-to-Primary 



Diameter 



(cm) 


Measured 


Computed 


4 


0.54 


0.50 


6 


0.65 


0.63 


10 


0.80 


0.79 


14 


0.86 


0.86 



RESULTS 

Compared in Table 1 are the scatter-to-primary measured by Barnes 
and Brezovich and the computed values for a 6 cm Lucite phantom and 
different field sizes. The agreement between the two is excellent. Good 
agreement (1-5%) was also obtained between the depth dose 
measurements of Hammerstein, et al and the computed values. 

A paper containing a comprehensive tabulation of our calculation of 
normalized average glandular doses for different breast thicknesses, 
compositions, tube potentials and HVLs has been submitted for 
publication^ 1 4 h Results from this tabulation for a 50% glandular - 50% 



762 



adipose 5 cm breast are listed in Table 2 for tube potentials of 25, 30 and 35 
kVp and 0 and 3.5 mm of Lexan added to the beam. The latter is typical of 
the maximum thickness of compression paddles on state-of-the-art 
mammography units. Of interest is that for a given HVL, D gN increases with 
increasing kVp. 



Table 2. Computed normalized average glandular dose (rad/R) for 50% 
glandular - 50% fat 5 cm breast 



Tube Potential 
(kVp) 


Lexan Thickness 
(mm) 


HVL 
(mm of AI) 


D gN 
(rad/R) 


25 


0 
3.5 


0.266 
0.331 


114 
138 


30 


0 
3.5 


0.306 
0.374 


135 
160 


35 


0 

3.5 


0.332 
0.400 


149 
173 



Table 3. Normalized average glandular doses for 
molybdenum target source assembly interpolated from 
FDA tabulation 



HVL 


DgN 


(mm of AI) 


(rad/R) 


0.26 


126 


0.28 


133 


0.30 


140 


0.32 


147 


0.34 


154 


0.36 


161 
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DISCUSSION 

The agreement between our computed scatter-to-primary and depth 
dose values and experimental measurements gives us a high degree of 
confidence in our Monte Carlo simulation techniques and in our normalized 
average glandular dose results. For comparison Table 3 list values for a 
molybdenum target x-ray tube that were extrapolated from the FDA 
tabulation. For a given HVL, these values agree quite well with our 35 kVp 
results. However, they are 10% greater than our 25 kVp results. Tube 
potentials of from 25 to 28 kVp are commonly used in mammography. At 
these potentials our results indicate that the breast doses calculated with 
the FDA tabulation typically over estimates the dose from 5 to 10%. 

Our results include the effect of x-ray tube potential and are therefore 
an improvement on the accuracy with which one can calculate average 
glandular dose in mammography. Our strategy in computing the average 
glandular dose is different in several respects from the earlier work of 
Rosenstein and Warner^ on which the FDA average glandular dose tables 
are based. First, they neglected to take into account the coherent scattering 
and electron-binding effects for coherent and incoherent scattering in their 
simulation. These effects could result in errors of up to 20% for the depth 
dose, and 5% for Dg^. Second, Rosenstein and Warner's computations 
utilized coarse energy resolution — 7 photon energies ranging from 1 0 keV 
to 55 keV compared to 28 energies ranging from 7.5 keV to 34.5 keV. 
Third, Rosenstein and Warner did not employ spectra that are consistent 
with modern clinical practice. They were interested in a wider range of beam 
qualities than are currently employed in screen-film mammography. As a 
result their compilation provides an average estimate of DgN for a given HVL 
for molybdenum target, glass window and molybdenum-tungsten alloy, 
beryllium window source assemblies with different added filtrations and tube 
potentials. The spectra we employed are consistent with state-of-the-art 
screen-film mammography. 
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1. INTRODUCTION 

Video display technology is recognized as the limiting step 
in the imaging chain of a workstation. Measurement of the 
image quality of video displays therefore deserves further 
attention. One approach is to measure the physical 
characteristics of the monitor with, for example, 
photometric measurements!? 2 . Unfortunately, this type of 
measurement does not allow for perceptual issues involved 
in diagnosis. Another approach is to use test patterns (e.g., 
SMPTE test pattern^) and determine, for example, the 
limiting resolution of the monitor. This type of test is 
subjective and difficult to quantify; furthermore, no 
consideration is given to the observer's confidence level or 
the false positive problem. 

Receiver Operating Characteristic (ROC) methodology has 
been used to measure the efficiency of human observers 
viewing video display s*. This type of measurement could, in 
principle, be used to quantify the performance of displays. 
However, these ROC experiments are quite cumbersome to 
perform, needing many images to build up acceptable 
statistical precision. 

An alternate type of observer performance experiment, the 
Free-response Receiver Operating Characteristic (FROC) 
experiment, has recently been studied5>6>7. Unlike the ROC 
experiment which must ignore the location information in 
detection/location experiments, and are limited to 
experiments with 0 or 1 abnormality per image, the FROC 
experiment requires correct location for a true positive 
event, and allows for multiple signals and observer 
responses per image. It is recognized that FROC methodology 
is intrinsically capable of better statistical precision for a 
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given number of images than ROC. In this study we present 
preliminary FROC measurements which extract an index of 
performance to good accuracy using relatively few images. 
We also suggest a new way of objectively evaluating the 
image quality of displays. 



2. METHODS 

We employed a Virtual Imaging workstation (VI-2000) with a 
1280 x 1024 x 8 bits display for the experiments. We 
generated images consisting of signals (discs of 4 pixel 
radius, or dots of 1 pixel) embedded in noise using a 
computer and a random number generator. Each image 
contained 5-12 non-overlapping signals, and typically 30 to 
60 images were generated. The pixel values inside the signal 
were generated from a gaussian distribution with mean |i s 
and standard deviation a gray scale units (gsu), while the 
pixel values outside the discs were generated from a 
gaussian distribution with mean \i n (=110 gsu) and standard 
deviation a. Values used for Aji (= \i s - p, n ) and a are tabulated 

in the table. Neglecting MTF effects, the Signal to Noise Ratio 
(SNR) is given by 

SNR=^Vn~ Eq. 1 

where n is the total number of pixels associated with each 
object ( n=49 for the discs, and n=l for the dots). Two 
observers were used for this preliminary study. They 
interacted with the images using a 'mouse 1 to move a pointer 
on the screen. They indicated perceived signal objects (discs 
or dots) by moving the pointer to the center of the object 
and clicking the 'mouse* button. The computer then drew a 
box around the indicated position. At the end of each image 
the observers had the option to see the locations of all signal 
objects present in the image (feedback). Data was collected 
in the Free-response Forced Error (FFE) manner?, i.e., the 
observer did not indicate confidence levels but instead 
indicated the most likely disc location first, followed by the 
next most likely one, and so on. The computer kept a record 
of all responses, and determined that the observer had made 
a false positive error before allowing him to move on to the 
next image (actually, to allow flexibility in subsequent 
analysis, we required the observers to make three false 
positive errors on each image). 
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The response data was analyzed in the following manner. 
The criterion for a true positive was arbitrarily set at a 6 
pixel maximum separation between the coordinates of the 
center of an object and the coordinates indicated by the 
reader. The analysis consisted of counting the number of 
hits before the first false positive, and dividing by the 
number of signals present in the image. This is an estimate 

of the FROC area parameter for that image (Aj). Averaging 

the Aj over all images yields A\ for the image set. The 
standard deviation is calculated in the usual manner, namely 
the standard deviation of the Aj divided by the square root of 
the number of images. 



3. DISCUSSION 

The table summarizes the results of this study. The column 
under 'Contrast 1 refers to whether or not the observers were 
allowed to use the window and level controls to optimize the 
display. The column under Ai lists the FROC index (which 
can take on values in the range 0.0 to 1.0, unlike the ROC 
index which ranges from 0.5 to 1.0) and the corresponding 
standard deviation. It is seen that when the noise level a is 
decreased, while maintaining a constant SNR, the 
performance of the observers decreases. This is consistent 
with the noise introduced by the display becoming more of a 
limitation than the intrinsic (computer generated) noise 
level of the pixels. This suggests a way to measure the 
display noise as discussed below. 

It is seen from the table that the standard deviation of the 
FROC index is typically 6 to 9% of the Aj value. It should be 
borne in mind that these are derived from as few as 30 
images. The standard deviations can be reduced further by 
using more images, but the number of images needed is still 
much smaller than that needed in studies using ROC 
experiments. This is because the FROC experiment utilizes 
information (location data, and multiple response data) that 
ROC analysis is forced to discard. 
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The preliminary measurements described here suggest a 
strategy to measure the image quality of displays: we 
construct 'calibration* images at different SNR's, where the 
a values are intentionally chosen to be large (A|x should be 
correspondingly large), so that display noise is not a 
limitation. Measuring the performance of an observer over 
these images yields an empirical 'calibration* curve for the 
observer, i.e., a plot of Aj vs. SNR (Fig. 1). Next we prepare a 
test set of images for which a = 0, so that any noise 
introduced is due to the display. By selecting A \i 
appropriately, perhaps a few grey levels, one can arrange to 
have the signals neither too obvious nor too difficult. 
Measuring the performance of the observer using the a = 0 
image set and using the calibration curve, we can convert 
the measured Ai(0) to an effective Signal to Noise Ratio, 
SNR(O), as depicted in Fig. 1. It can be seen that, to first 
order, variability in observers cancels out when SNR(O) is 
calculated in this manner. A 'better 1 observer will yield 
higher Aj values than a 'poorer* observer, but the SNR(O) 
for each will be the same because they will each have a 
unique calibration curve. 

The SNR(O) can be converted to an equivalent noise, a e q, by 
inverting Eq. 1, i.e., 

The equivalent noise can be thought of as that computer 
generated noise on a perfect display that yields the same 
performance for a given reader as the actual display noise. 
The equivalent noise can be related to a noise-power 
spectrum to normalize dependences on pixel dimensions, etc. 
The noise measurement could be repeated for different size 
discs/dots, and at different average grey-scale levels, to 
more fully characterize the display. Obvious applications of 
the methodology suggested in this paper would be to 
evaluating the newer high resolution displays that are 
becoming available. Finally, we note that the method need 
not be restricted to video displays and could be applied, with 
few modifications, to evaluating digital harcopy devices. 
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5. TABLE: SUMMARY OF RESULTS 



Obj. 




a 


SNR 


Cont. Read. 


A i(s.d.) 


Disk 


20 


25 


5.60 


Yes 


A 


0.589 (0.033) 


Disk 


20 


25 


5.60 


Yes 


B 


0.559 (0.036) 


Disk 


4 


5 


5.60 


No 


A 


0.353 (0.032) 


Disk 


4 


5 


5.60 


No 


B 


0.436 (0.036) 


Dot 


18 


3 


6.00 


No 


A 


0.671 (0.038) 


Dot 


18 


3 


6.00 


No 


B 


0.553 (0.040) 



1.0 




SNR 

Fig. 1. Illustrating the proposed method for measuring 
display noise. The curve sketched is a Calibration 1 
curve for an observer measured empirically. It is 
measured in the large computer generated noise limit 
(so that display noise is not a limitation). Aj (0) is 
measured with zero computer generated noise, so that 
any noise introduced is due to the display. SNR(O) can 
be converted to an effective display noise as explained 
in the text. 
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